March  9,  2007 


Report  No.  D-2007-066 


Jnspector  (^ene 

United  States 
Department  of  Defense 


Navy  Acquisition  Executive's  Management 
Oversight  and  Procurement  Authority  for 
Acquisition  Category  I  and  II  Programs 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

09  MAR  2007  2' REPORT  TYPE 

3.  DATES  COVERED 

00-00-2007  to  00-00-2007 

4.  TITLE  AND  SUBTITLE 

Navy  Acquisition  Executive’s  Management  Oversight  and  Procurement 
Authority  Category  I  and  II  Programs 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

ODIG-AUD  Department  of  Defense  Inspector  General, 400  Army  Navy 
Drive  Suite  801, Arlington, VA, 22202-4704 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

102 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Additional  Copies 

To  obtain  additional  copies  of  this  report,  visit  the  Web  site  of  the  Department  of 
Defense  Inspector  General  at  http://www.dodig.mil/audit/reports  or  contact  the 
Secondary  Reports  Distribution  Unit  at  (703)  604-8937  (DSN  664-8937)  or  fax 
(703)  604-8932. 

Suggestions  for  Future  Audits 

To  suggest  ideas  for  or  to  request  future  audits,  contact  the  Office  of  the  Deputy 
Inspector  General  for  Auditing  at  (703)  604-9142  (DSN  664-9142)  or  fax  (703) 
604-8932.  Ideas  and  requests  can  also  be  mailed  to: 

ODIG-AUD  (ATTN:  Audit  Suggestions) 

Department  of  Defense  Inspector  General 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-4704 


DEPARTMENT  OF  DEFENSE 


i  ivi 


To  report  fraud,  waste,  mismanagement,  and  abuse  of  authority. 

Send  written  complaints  to:  Defense  Hotline,  The  Pentagon,  Washington,  DC  20301-1900 
Phone:  800.424.9098  e-mail:  hotline@dodig.mil  www.dodig.mil/hotline 


Acronyms 


ACAT 

AoA 

ASN(RD&A) 

ASW 

CLIP 

FAR 

GAO 

IG 

LRIP 

NAE 

ORD 

OSCAR 

PEO 

SECNAV 

SEWIP 

USD(AT&L) 


Acquisition  Category 
Analysis  of  Alternatives 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition) 

Antisubmarine  Warfare 

Common  Link  Integration  Processing 

Federal  Acquisition  Regulation 

Government  Accountability  Office 

Inspector  General 

Low-Rate  Initial  Production 

Navy  Acquisition  Executive 

Operational  Requirements  Document 

Open  Systems  Core  Avionics  Requirements 

Program  Executive  Officer 

Secretary  of  the  Navy 

Surface  Electronic  Warfare  Improvement  Program 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-4704 


MEMORANDUM  FOR  NAVAL  INSPECTOR  GENERAL 


March  9, 2007 


SUBJECT:  Report  on  Navy  Acquisition  Executive’s  Management  Oversight  and 

Procurement  Authority  for  Acquisition  Category  I  and  II  Programs  (Report 
No.  D-2007-066) 


We  are  providing  this  report  for  review  and  comment.  We  considered 
management  comments  on  a  draft  of  this  report  in  preparing  the  final  report. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 

As  a  result  of  management  comments,  we  revised  Recommendation  A.3.  and  deleted 
Recommendation  A.  1  .e.  Recommendation  B.  1 .  remains  unresolved.  In  addition,  we 
added  Recommendation  B.2.  and  deleted  Recommendation  B.6.  We  renumbered  B.2., 
B.3.,  and  B.4.  as  B.3.,  B.4.,  and  B.5.  accordingly.  We  revised  Recommendations  B.3.  and 
B.4.  We  request  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  provide  comments  on  Recommendations  A.3.,  B.L,  B.2.,  and  B.4.  by 
April  9, 2007. 

If  possible,  please  send  management  comments  in  electronic  format  (Adobe 
Acrobat  file  only)  to  AudACM@dodig.mil .  Copies  of  the  management  comments  must 
contain  the  actual  signature  of  the  authorizing  official.  We  cannot  accept  the  /  Signed  / 
symbol  in  place  of  the  actual  signature.  If  you  arrange  to  send  classified  comments 
electronically,  they  must  be  sent  over  the  SECRET  Internet  Protocol  Router  Network 
(SIPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Questions  should  be  directed 
to  Ms.  Jacqueline  L.  Wicecarver  at  (703)  604-9077  (DSN  664-9077)  or 
Mr.  Sean  A.  Davis  at  (703)  604-9049  (DSN  664-9049).  See  Appendix  I  for  the  report 
distribution.  The  team  members  are  listed  inside  the  back  cover. 

By  direction  of  the  Deputy  Inspector  General  for  Auditing: 

r- 

Richard  B.  Jolliffe 
Assistant  Inspector  General 
Acquisition  and  Contract  Management 


Department  of  Defense  Office  of  Inspector  General 
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Navy  Acquisition  Executive’s  Management  Oversight  and  Procurement 
Authority  for  Acquisition  Category  I  and  II  Programs 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  Civil  service  and  military  personnel 
involved  in  the  management  oversight  and  procurement  authority  for  Navy  acquisition 
programs  should  read  this  report.  It  discusses  program  management  oversight  issues  that 
the  Navy  should  address  to  improve  how  acquisition  officials  manage  and  acquire 
weapon  systems. 

Background.  This  is  the  third  in  a  series  of  reports  that  discusses  the  Service 
Acquisition  Executives’  management  oversight  and  procurement  authority  for 
Acquisition  Category  IC  and  II  programs.  This  report  discusses  the  management 
oversight  and  procurement  authority  within  the  Navy.  Two  other  reports  discussed  the 
management  oversight  and  procurement  authority  within  the  Anny  and  Air  Force.  We 
initiated  this  audit  due  to  congressional  and  DoD  interest  in  whether  milestone  decision 
authorities  and  procurement  officials  for  the  Services  were  complying  with  statutory  and 
regulatory  requirements  in  the  DoD  acquisition  process.  We  evaluated  the  adequacy  of 
the  Navy  Acquisition  Executive  (NAE)  management  oversight  and  procurement  authority 
by  reviewing  13  Acquisition  Category  IC  and  II  programs  with  a  total  estimated  research 
and  development  cost  of  $6. 170  billion  and  a  total  estimated  procurement  cost  of 
$28,244  billion  in  FY  2006  dollars.  The  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  is  the  milestone  decision  authority  for  the  development 
and  procurement  of  Navy  Acquisition  Category  IC  and  II  programs.  The  Navy  uses  a 
database  called  Dashboard  to  help  manage  the  Navy  programs. 

Results.  In  general,  the  Navy  implemented  the  management  controls  in  the  DoD  5000 
series  of  documents  and  the  Federal  Acquisition  Regulation.  We  did  not  find 
management  control  problems  like  those  identified  in  our  earlier  reviews  of  the  Boeing 
KC-767A  tanker  aircraft  and  the  C-130J  aircraft.  We  also  determined  that  the  NAE 
internal  controls  were  adequate.  We  identified  no  material  internal  control  weaknesses  in 
the  management  oversight  and  procurement  authority  for  Acquisition  Category  IC  and  II 
programs.  Specifically,  we  found  no  evidence  that  Navy  milestone  decision  authorities 
used  their  positions  to  inappropriately  influence  the  results  of  contractor  selection  and 
negotiations.  In  addition,  the  NAE  oversight  of  Acquisition  Category  IC  and  II  programs 
was  generally  adequate.  However,  management  oversight  of  areas  related  to  the 
documentation  requirements  in  support  of  program  milestone  decision  reviews,  test  and 
evaluation,  and  the  Dashboard  reporting  system  could  be  improved. 


Since  FY  2000,  the  NAE  approved  10  of  the  13  programs  we  reviewed  for  entry  into  the 
next  phase  of  the  acquisition  process  before  obtaining  all  required  or  properly  approved 
acquisition  documentation.  Additionally,  the  NAE  did  not  require  3  of  the  13  programs 
to  have  approved  and  updated  acquisition  program  baseline  documentation  between 
milestone  decision  reviews  when  significant  changes  affected  the  programs.  As  a  result, 
the  NAE  did  not  have  all  the  necessary  infonnation  to  make  fully  informed  milestone 
decisions  and  act  appropriately  between  milestone  decision  reviews.  Therefore,  we 
recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  comply  with  DoD  and  Secretary  of  the  Navy  policy  when  reviewing 
programs  for  milestone  reviews;  maintain  decision  documents  as  a  result  of  milestone 
reviews  in  the  acquisition  decision  memorandum;  track  deviation  reports  in  the  Navy 
Dashboard  database;  and  require  program  managers  to  maintain  approved  documentation 
for  the  life  of  the  acquisition  program  (finding  A). 

The  NAE  approved  8  of  the  13  programs  for  either  low-rate  initial  production  or  full-rate 
production.  The  NAE  approved  three  programs  for  low-rate  initial  production  before 
program  officials  conducted  an  operational  assessment.  The  NAE  approved  one  program 
for  low-rate  initial  production  even  though  operational  assessment  determined  the  system 
was  immature  and  critical  operational  issues  could  not  be  evaluated.  Further,  on  one 
program,  the  Navy  purchased  all  systems  prior  to  completing  operational  testing.  As  a 
result,  the  NAE  did  not  determine  whether  programs  were  operationally  effective  and 
suitable  prior  to  approving  production  quantities  costing  about  $25  billion.  This  could 
result  in  additional  development  work  and  operational  testing,  cost  overruns,  and 
schedule  delays. 

We  recommend  that  the  NAE  acquisition  decision  memorandums  include  justification  for 
exceeding  the  10-percent  low-rate  initial  production  guideline,  justification  for  approving 
low-rate  initial  production  or  full-rate  production  when  exit  criteria  are  not  met,  a 
statement  that  previously  approved  exit  criteria  have  been  deleted  or  revised,  and  an 
explanation  for  the  change  to  the  exit  criteria.  In  addition,  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition)  should  formally  document  the  rationale 
for  allowing  programs  to  proceed  through  milestone  decision  reviews  without  the 
required  testing  and  documentation  (finding  B). 

Program  managers  for  2  of  the  13  Navy  programs  did  not  update  Dashboard  infonnation 
on  a  quarterly  basis.  Additionally,  program  managers  for  evolutionary  acquisitions  could 
not  report  each  increment  separately  in  Dashboard.  This  practice  will  result  in  a  loss  of 
historical  data  for  past  increments.  Navy  officials  will  not  be  able  to  distinguish  between 
important  milestone  decision  dates  and  data  for  each  increment.  This  condition  could 
result  in  Navy  officials  being  unable  to  interpret  data  in  the  Dashboard  reports. 

Therefore,  we  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  issue  guidance  mandating  the  use  and  quarterly  update  of 
Dashboard  and  modify  Dashboard  to  distinguish  increments  for  evolutionary  acquisition 
programs  (finding  C). 

See  the  Findings  section  for  details  on  the  audit  results  and  recommendations. 

Management  Comments  and  Audit  Response.  We  received  comments  from  the 
Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget),  who  responded  on 
behalf  of  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition). 
He  generally  concurred  with  our  recommendations  on  preparing  and  updating  program 
documentation.  However,  he  nonconcurred  with  our  recommendation  that  the  AV-8B 
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Harrier  II  OSCAR  program  manager  develop  and  prepare  a  tailored  information  support 
plan.  The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that 
completing  an  information  support  plan  when  the  program  is  58-percent  delivered  would 
be  an  inefficient  use  of  available  resources  with  limited  or  no  benefit  to  the  program. 

In  response  to  management  comments,  we  added  an  additional  recommendation  and 
deleted  one  recommendation.  The  added  recommendation  is  for  the  Navy  to  update  the 
Navy  Acquisition  Guidebook  to  conform  to  the  requirements  in  DoD  Instruction  5000.2. 
In  addition,  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget) 
comments  were  not  responsive  on  two  of  the  seven  recommendations  on  operational 
testing  in  the  milestone  decision  process.  He  nonconcured  with  the  recommendation  to 
approve  a  program  office’s  request  to  exceed  the  10-percent  low-rate  initial  production 
limitation  only  with  the  Commander,  Operational  Test  and  Evaluation  Force 
recommendation  that  the  system  is  ready  to  proceed  to  low-rate  initial  production.  In 
response  to  management  comments,  we  revised  the  recommendation  to  state  that  the 
Assistant  Secretary  of  the  Navy  should  assess  the  cost  and  benefits  of  a  break  in 
production  versus  continuing  annual  buys  when  the  program  office  expects  to  exceed  the 
low-rate  initial  production  quantity  approved  at  Milestone  B.  Additionally,  management 
comments  were  not  responsive  to  the  recommendation  requiring  program  offices  to 
schedule  and  complete  an  operational  assessment  prior  to  the  low-rate  initial  production 
milestone  decision  in  accordance  with  DoD  Instruction  5000.2. 

Based  on  management  comments,  we  also  revised  the  recommendation  regarding 
documenting  in  the  acquisition  decision  memorandum  when  quantities  exceed  the 
10-percent  low-rate  initial  product  guideline. 

The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  concurred  with 
both  of  our  recommendations  on  the  Dashboard  database.  He  stated  that  official 
Dashboard  guidance  will  be  issued  and  that  Dashboard  will  be  modified  to  distinguish 
different  increments  for  evolutionary  programs.  The  comments  were  responsive  to  the 
rec  ommendations . 

We  request  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  provide  comments  on  Recommendations  A. 3.,  B.I.,  B.2.,  and  B.4.  by 
April  9,  2007.  See  Appendix  H  for  a  discussion  of  management  comments  on  the 
findings.  Also,  see  the  Findings  section  of  the  report  for  a  discussion  of  management 
comments  on  the  recommendations  and  the  Management  Comments  section  of  the  report 
for  the  complete  text  of  the  comments. 
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Background 


This  report  is  the  third  in  a  series  of  audit  reports  that  discusses  the  Service 
Acquisition  Executives’  management  oversight  and  procurement  authority  for 
Acquisition  Category  (AC AT)  I  and  II  programs.  This  report  discusses  Navy’s 
management  oversight  and  procurement  authority.  Two  other  reports  discussed 
the  Army  and  Air  Force  management  oversight  and  procurement  authority.  We 
initiated  this  audit  because  of  congressional  and  DoD  interest  in  whether  Service 
milestone  decision  authorities  and  procurement  officials  were  complying  with  the 
statutory  and  regulatory  requirements  for  acquisitions.  Previous  audits  of  the 
Boeing  KC-767A  tanker  aircraft  and  the  C-130J  aircraft  found  that  the  program 
offices  failed  to  implement  management  controls  in  accordance  with  the  policy 
and  guidance  in  the  DoD  5000  series  of  documents  and  in  the  Federal  Acquisition 
Regulation  (FAR).  See  Appendix  D  for  a  comparison  of  conditions  identified 
during  three  Boeing  KC-767A  and  C-130J  audits  with  the  13  Navy  programs 
reviewed  during  this  audit. 

The  DoD  5000  series  of  documents  provides  policy  and  guidance  to  the 
DoD  Components  to  manage  their  acquisition  programs.  The  Defense 
Acquisition  Executive  is  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Fogistics  (USD[AT&FJ)  who  supervises  the  Defense 
Acquisition  System.  The  USD(AT&F)  designates  programs  as  either  AC  AT  ID 
or  ACAT  IC,  depending  on  the  required  level  of  oversight.  The  USD(AT&F)  is 
the  milestone  decision  authority  for  ACAT  ID  programs.  The  head  of  the  DoD 
Component  or,  if  delegated,  the  DoD  Component  Acquisition  Executive  is  the 
milestone  decision  authority  for  ACAT  IC  programs. 

Secretary  of  the  Navy  (SECNAV)  Instruction  5000. 2C,  “Implementation  and 
Operation  of  the  Defense  Acquisition  System  and  the  Joint  Capabilities 
Integration  and  Development  System,”  November  19,  2004,  implements  the 
DoD  5000  series  of  documents.  SECNAV  Instruction  5000. 2C  states  that  the 
Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 
(ASN[RD&A])  is  the  Department  of  the  Navy  Component  Acquisition  Executive. 
The  ASN(RD&A),  also  known  as  the  Navy  Acquisition  Executive  (NAE),  is  the 
milestone  decision  authority  for  ACAT  IC  and  II  programs  and  may  delegate  this 
responsibility  as  authorized  by  law  and  regulations.  The  NAE  designates  Program 
Executive  Officers  (PEO)  for  executive  management  of  assigned  acquisition 
programs  and  assigns  program  managers  to  execute  the  acquisition  programs  in 
accordance  with  approved  cost,  schedule,  and  performance  thresholds  set  in  the 
acquisition  program  baseline.  The  Secretary  of  the  Navy,  ASN(RD&A),  Chief  of 
Naval  Operations,  Systems  Commands,  PEOs,  and  program  managers  use  an 
automated  tool  called  Dashboard  to  manage  the  various  ACAT  programs  with 
consistent  data  throughout  the  chain  of  command. 

ACAT  I  (IC  and  ID)  programs  are  major  Defense  acquisition  programs  with  an 
estimated  total  expense  for  research,  development,  test,  and  evaluation  of  more 
than  $365  million  in  FY  2000  constant  dollars  or,  for  procurement,  of  more  than 
$2,190  billion  in  FY  2000  constant  dollars.  ACAT  II  programs  are  major  systems 
with  an  estimated  total  expense  for  research,  development,  test,  and  evaluation  of 
less  than  $365  million  but  more  than  $140  million,  or  for  procurement,  of  less 
than  $2,190  billion  but  more  than  $660  million  in  FY  2000  constant  dollars. 
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We  evaluated  the  adequacy  of  the  NAE  management  oversight  and  procurement 
authority  by  reviewing  13  AC  AT  IC  and  II  programs  with  a  total  estimated 
research  and  development  cost  of  $6,170  billion  and  a  total  estimated 
procurement  cost  of  $28,244  billion  in  FY  2006  dollars.  Appendix  E  shows  an 
overview  of  the  programs  reviewed  including  the  individual  funding  for  each  of 
the  programs.  Of  the  13  programs  selected  for  review,  5  programs  were  in  the 
system  development  and  demonstration  phase,  and  8  programs  were  in  the 
production  phase  of  the  acquisition  process.  In  addition,  8  of  the  13  programs 
were  listed  on  the  Director,  Operational  Test  and  Evaluation  Oversight  list. 
Appendix  C  contains  a  glossary  of  technical  terms  used  in  this  report. 


Objective 


The  overall  audit  objective  was  to  evaluate  whether  the  NAE  management 
oversight  and  procurement  authority  for  ACAT  I  and  II  programs  was  adequate. 
Specifically,  the  audit  evaluated  the  adequacy  of  the  program  management  and 
procurement  decision  process  used  by  the  NAE.  We  also  reviewed  management 
controls  as  they  related  to  the  audit  objective.  See  Appendix  A  for  a  discussion  of 
the  scope  and  methodology  and  Appendix  B  for  prior  audit  coverage  related  to  the 
audit  objectives.  See  Appendix  F  for  a  description  of  the  13  Navy  acquisition 
programs.  See  Appendix  G  for  a  discussion  of  other  matters  of  interest 
concerning  Acquisition  Coordination  Teams  and  documentation  of  the  contract 
negotiations. 


Review  of  Internal  Controls 


The  NAE  internal  controls  over  the  management  and  procurement  of  ACAT  IC 
and  II  programs  were  adequate. 
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A.  Preparing  and  Updating  Program 
Documentation 

Since  FY  2000,  the  NAE  approved  10  of  the  13  programs  we  reviewed  for 
entry  into  the  next  phase  of  the  acquisition  process  before  obtaining  all 
required  or  properly  approved  acquisition  documentation.  The  NAE  did 
not  require  3  of  the  13  programs  to  have  approved  and  updated  acquisition 
program  baseline  documentation  between  milestone  decision  reviews 
when  significant  changes  affected  the  programs.  These  conditions 
occurred  because  the  NAE  did  not  comply  with  DoD  and  Navy  acquisition 
policy  when  reviewing  and  approving  programs  for  a  milestone  decision 
review  and  because  the  NAE  was  not  tracking  program  deviation  reports 
in  the  Navy  Dashboard  database.  In  addition,  it  was  unclear  why  the  NAE 
allowed  programs  to  proceed  through  milestones  without  required  or 
properly  approved  acquisition  documentation.  No  formal  tailored 
documentation  agreements  existed  and  documentation  decisions 
associated  with  each  milestone  review  were  not  being  formally 
documented  in  document  waivers,  tailoring  agreements,  or  acquisition 
decision  memorandums.  As  a  result,  the  NAE  did  not  have  all  the 
necessary  information  required  to  make  fully  informed  milestone 
decisions  and  to  take  appropriate  management  actions  between  milestone 
decision  reviews. 


DoD  and  Navy  Policy  Applicable  to  the  Acquisition  Process 

DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition  System,” 

May  12,  2003,  and  SECNAV  Instruction  5000.2C  establish  requirements  for 
preparing,  updating,  and  obtaining  required  program  documentation.  Additional 
policy  has  been  issued  for  program  documentation  but  has  not  yet  been 
incorporated  in  DoD  Instruction  5000.2  or  SECNAV  Instruction  5000.2C.  The 
USD(AT&L)  memorandum,  “Policy  for  Systems  Engineering  in  DoD,”  issued 
February  20,  2004,  establishes  the  requirements  for  developing  systems 
engineering  plans.  The  Defense  Acquisition  Guidebook  provides  the  acquisition 
community  with  a  guide  to  best  practices  for  Defense  acquisitions.  Two 
additional  SECNAV  Instructions  provide  guidance  on  Navy  policy,  processes, 
and  acquisition  responsibilities  associated  with  life-cycle  management. 

DoD  Instruction.  DoD  Instruction  5000.2  specifies  program  documents  that 
program  managers  must  provide  at  program  milestone  reviews.  This  guidance 
establishes  a  simplified  and  flexible  management  framework  for  translating 
approved  mission  needs  and  technology  opportunities  into  stable,  affordable,  and 
well-managed  acquisition  programs  that  include  weapon  systems  and  automated 
information  systems. 

USD(AT&L)  memorandum,  “Policy  for  Systems  Engineering  in  DoD,” 

February  20,  2004,  requires  “all  programs  responding  to  a  capabilities  or 
requirements  document,  regardless  of  acquisition  category,  to  apply  a  robust 
systems  engineering  approach  that  balances  total  system  perfonnance  and  total 
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ownership  costs  within  the  family-of-systems,  systems-of-systems  context.”  The 
memorandum  requires  program  managers  “to  develop  a  systems  engineering  plan 
for  NAE  approval  in  conjunction  with  each  milestone  review.”  The  systems 
engineering  plan  should  also  be  integrated  with  the  acquisition  strategy.  The 
systems  engineering  plan  must  describe  the  program’s  overall  approach,  including 
processes,  resources,  metrics,  and  applicable  performance  incentives.  It  must  also 
state  the  timing,  conduct,  and  success  criteria  of  technical  reviews. 

Navy  Instructions.  SECNAV  Instruction  5000. 2C  identifies  milestone 
documentation  requirements  as  the  key  management  control  for  acquisition 
programs  and  the  milestone  decision  review  process  as  the  evaluation  of  that 
control. 

SECNAV  Instruction  5400. 15B,  “Department  of  the  Navy  Research, 
Development,  and  Acquisition,  and  Associated  Life-Cycle  Management 
Responsibilities,”  December,  23,  2005,  describes  the  relationships  between  the 
ASN(RD&A),  PEOs,  and  the  Chief  of  Naval  Operations  and  associated  life-cycle 
management  responsibilities.  It  also  documents  previously  established  duties  and 
responsibilities  of  the  ASN(RD&A). 

SECNAV  Instruction  5420. 188F,  “Acquisition  Category  Program  Decision 
Process,”  November  2,  2005,  provides  policy  and  process  for  making  Navy 
ACAT  program  decisions  and  outlines  managers’  responsibilities  for  the 
oversight  process.  See  the  Management  Oversight  section  for  additional 
oversight  responsibilities. 

Defense  Acquisition  Guidebook.  The  Defense  Acquisition  Guidebook  provides 
the  acquisition  community  with  best  practices  for  Defense  acquisitions.  The 
Guidebook  is  a  practical  reference  that  supports  acquisition  decision  makers  in 
effectively  fulfilling  their  management  duties.  The  content  includes  general 
policies  and  procedures  that  complement  the  statutory  and  regulatory 
requirements  outlined  in  the  DoD  5000  series  of  guidance.  It  also  includes 
instruction  on  program  management  responsibilities,  developing  acquisition  goals 
and  strategies,  and  preparing  acquisition  documentation. 


Management  Oversight 


The  management  oversight  responsibilities  for  the  Navy  program  decision  process 
are  set  forth  in  SECNAV  Instruction  5000. 2C,  SECNAV  Instruction  5400. 15B, 
and  SECNAV  Instruction  5420. 188F.  The  levels  of  management  oversight  and 
responsibilities  are  defined  below. 

Milestone  Decision  Authority.  SECNAV  Instruction  5000.2C  and  SECNAV 
Instruction  5420. 188F  designates  the  ASN(RD&A)  as  the  NAE.  The  NAE  is 
responsible  for  all  research,  development,  and  acquisition  conducted  for  the 
Department  of  the  Navy.  The  NAE  serves  as  the  milestone  decision  authority  for 
ACAT  IC  and  II  programs.  The  milestone  decision  authority  is  required  to 
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conduct  milestone  reviews  for  all  assigned  Department  of  the  Navy  ACAT 
programs  and  holds  a  program  decision  meeting  before  milestone  decisions.  The 
program  decision  meeting  is  the  NAE  milestone  review  forum.  The  primary 
duties  of  the  NAE  include  reviewing  and  approving  all  appropriate  milestone 
documentation,  reviewing  the  program  decision  briefing,  chairing  the  program 
decision  meeting,  and  signing  the  acquisition  decision  memorandum. 

Deputy  Assistant  Secretaries  of  the  Navy.  The  Deputy  Assistant  Secretaries  of 
the  Navy  for  ACAT  IC  and  II  programs  are  responsible  for  cochairing  or 
appointing  an  action  officer1  to  cochair  the  Acquisition  Coordination  Team.  The 
cochair  ensures  that  milestone  documentation  is  complete  and  has  been  submitted 
to  appropriate  review  and  approval  authorities  prior  to  scheduling  a  program 
decision  meeting,  and  prepares  and  obtains  a  signature  on  the  acquisition  decision 
memorandum.  As  of  October  2006,  10  Deputy  Assistant  Secretaries  of  the  Navy 
oversee  specific  functional  areas  within  the  Office  of  the  ASN(RD&A).  Each 
acquisition  program  is  assigned  a  Deputy  Assistant  Secretary  of  the  Navy  based 
on  the  program  type. 

Program  Executive  Officers.  PEOs  exercise  the  authority  of  the  NAE  to 
directly  supervise  the  management  of  assigned  programs,  maintaining  oversight 
of  cost,  schedule,  and  performance.  PEOs  are  responsible  for  all  aspects  of  life- 
cycle  management  for  their  assigned  programs.  PEOs  report  directly  to  the  NAE 
for  all  matters  pertaining  to  acquisition.  The  PEO  also  establishes  the  Acquisition 
Coordination  Team,  ensures  all  program  issues  have  been  addressed,  chairs  the 
acquisition  program  briefings,  and  notifies  the  Deputy  Assistant  Secretary  of  the 
Navy  to  schedule  the  program’s  milestone  review. 

Program  Managers.  Program  managers  cochair  the  Acquisition  Coordination 
Team,  develop  an  overall  approach  to  conduct  the  milestone  review,  prepare  and 
present  the  program  decision  briefing,  and  ensure  that  program  officials  obtain 
appropriate  reviews  and  approvals  for  acquisition  documentation.  Before 
program  initiation  and  before  the  NAE  approves  subsequent  milestone  reviews, 
program  managers  must  provide  mandatory  program  documents  for  the  milestone 
decision  authority  to  review. 


Submitting  Required  Program  Documentation  at  Milestone 
Decision  Reviews 


We  reviewed  13  Navy  acquisition  programs  that  had  milestone  reviews  held  with 
the  NAE  for  entry  into  the  system  development  and  demonstration  phase  and 
production  and  deployment  phase  of  the  acquisition  process.  The  NAE  approved 
10  of  the  13  programs  for  entry  into  the  next  acquisition  phase  even  though 
required  acquisition  documentation  was  not  provided  or  was  not  properly 
approved  at  the  time.  The  Deputy  Assistant  Secretary  of  the  Navy  assigned  to  the 
program  prepares  the  acquisition  decision  memorandum  and  the  NAE  approves  it. 
It  is  the  formal  approval  of  the  program  to  enter  the  next  acquisition  phase.  The 


'Action  officers  are  assigned  to  a  Deputy  Assistant  Secretary  of  the  Navy  and  work  closely  with  program 
management  to  ensure  that  milestone  documentation  requirements  are  met. 
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following  table  shows  whether  the  program  managers  for  the  13  Navy  programs 
could  provide  support  that  they  developed  8  mandatory  documents  and  had  each 
document  properly  approved  according  to  the  guidance. 

Key  Documentation  of  the  13  Navy  Programs 


Programs 

AoA 

APB 

Acquisition 

Strategy 

C4ISP 
or  ISP 

MNS 
or  ICD 

ORD, 
CDD, 
or  CPD 

SEP 

TEMP 

Advanced  Arresting  Gear 

Yes 

Yes 

Yes 

N/A1 

Yes 

Yes 

Yes 

Yes 

Advanced  Deployable 

System 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Active  Electronically 

Scanned  Array 

No 

Yes 

No 

Yes 

N/A2 

Yes 

Yes 

Yes 

Airborne  Laser  Mine 

Detection  System 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

AN/AQS-20A 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

AN/SPY- 1D(V) 

No 

Yes 

No 

N/A 

Yes 

Yes 

N/A3 

Yes 

Common  Link  Integration 
Processing 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

MH-60R 

No 

Yes 

Yes 

Yes 

No4 

Yes 

Yes 

No 

Mode  5 

No 

Yes 

No 

No 

Yes 

Yes 

N/A 

Yes 

Navy  Multiband  Terminal 

Yes 

No 

Yes 

Yes 

No 

Yes 

N/A3 

No 

Open  Systems  Core  Avionics 
Requirements 

Yes 

Yes 

Yes 

No 

No 

No 

N/A3 

Yes 

Surface  Electronic  Warfare 
Improvement  Program 

Yes 

Yes 

No 

N/A5 

Yes 

Yes 

No 

Yes 

Tactical  Tomahawk 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

N/A3 

Yes 

AoA  Analysis  of  Alternatives 

APB  Acquisition  Program  Baseline 

CDD  Capabilities  Development  Document 

CPD  Capabilities  Production  Document 

C4ISP  Command,  Control,  Communications,  Computers,  and  Intelligence  Support  Plan 

ISP  Information  Support  Plan 

ICD  Initial  Capabilities  Document 

MNS  Mission  Needs  Statement 

ORD  Operational  Requirements  Document 

SEP  Systems  Engineering  Plan 

TEMP  Test  and  Evaluation  Master  Plan 

1  Program  had  an  approved  ISP  waiver. 

2  Program  was  not  required  to  develop  a  mission  needs  statement. 

3  Not  required  to  develop  an  SEP  before  February  20,  2004. 

4  Program  used  the  ORD  to  show  mission  needs. 

5  Assistant  Secretary  of  Defense  (Networks  &  Information  Integration)  deferred  ISP  requirement  to  next 

milestone. 
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Analysis  of  Alternatives.  An  analysis  of  alternatives  (AoA)  is  the  evaluation  of 
the  operational  effectiveness,  operational  suitability,  and  estimated  costs  of 
alternative  systems  to  meet  a  mission  capability.  The  analysis  assesses  the 
advantages  and  disadvantages  of  alternative  systems  being  considered  to  satisfy  a 
validated  need,  including  the  sensitivity  of  each  alternative  to  possible  changes  in 
key  assumptions  and  variables.  SECNAV  Instruction  5000.2C  requires  an  AoA 
be  developed  for  both  ACAT  IC  and  II  programs  prior  to  entry  into  the  DoD 
acquisition  process.  According  to  the  Office  of  the  Secretary  of  Defense  Cost 
Analysis  Improvement  Group’s  “Operating  and  Support  Cost-Estimating  Guide,” 
May  1,  1992,  a  cost  and  operational  effectiveness  analysis  should  be  conducted  to 
assess  the  operational  effectiveness  and  suitability  of  proposed  concepts  at 
Milestones  I  and  II.2  The  cost  and  operational  effectiveness  analysis  considered 
the  cost-effectiveness  of  the  recommended  approach  and  of  alternative 
approaches.  Because  some  of  the  programs  were  initiated  when  different 
guidance  was  in  effect,  we  accepted  a  cost  and  operational  effectiveness  analysis 
in  lieu  of  an  AoA. 

Of  the  1 3  programs  chosen  for  review,  9  had  an  AoA  or  a  cost  and  operational 
effectiveness  analysis  prepared.  Seven  of  the  nine  documents  were  properly 
approved  by  NAE  or  did  not  require  NAE  approval  at  the  time  they  were  created; 
one  of  the  nine  documents  did  not  have  the  signature  page;  and  one  of  the  nine 
was  approved  by  the  PEO  rather  than  the  NAE. 

Program  officials  for  the  following  four  programs  were  unable  to  provide  an  AoA 
or  cost  and  operational  effectiveness  analysis: 

•  MH-60R  Multi-Mission  Helicopter, 

•  Tactical  Tomahawk, 

•  AN/SPY- 1D(V)  Radar  Upgrade,  and 

•  Mark  XIIA  Mode  5  Identification  Friend  or  Foe  Upgrade. 

As  a  result,  we  could  not  verify  that  the  AoAs  or  cost  and  operational 
effectiveness  analyses  existed  for  those  four  programs. 

AN/APG-79  Active  Electronically  Scanned  Array  Radar  System.  An 

AoA  was  provided  for  the  AN/APG-79  Active  Electronically  Scanned  Array 
Radar  System;  however,  the  signature  page  could  not  be  provided.  As  a  result, 
we  could  not  verify  that  the  AoA  had  been  approved  by  the  NAE. 

Common  Link  Integration  Processing.  SECNAV  Instruction  5000. 2C 
requires  the  AoA  be  approved  by  the  NAE,  the  Chief  of  Naval  Operations,  or  the 


2  Milestone  I  is  a  legacy  milestone  phase  in  which  the  program,  if  approved,  would  have  been  established 
as  a  new  acquisition  program  and  entered  into  the  program  definition  and  risk  reduction  phase  of  the 
legacy  acquisition  life  cycle.  It  is  the  equivalent  of  Milestone  A  under  the  current  acquisition  life  cycle. 
Milestone  II  is  a  legacy  milestone  phase  in  which  the  program,  if  approved,  would  have  been  approved  to 
enter  the  engineering  and  manufacturing  development  phase  of  the  legacy  acquisition  life  cycle.  It  is  the 
equivalent  of  Milestone  B  under  the  current  acquisition  life  cycle.  This  milestone  did  not  approve  low- 
rate  initial  production  for  the  program,  LRIP  was  approved  later  in  the  same  phase  as  its  own  decision. 
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Commandant  of  the  Marine  Corps,  but  makes  no  mention  of  the  PEO  being 
approved  to  sign  the  AoA.  The  AoA  was  approved  by  the  PEO  instead  of  the 
NAE  as  required  by  the  SECNAV  Instruction  5000. 2C.  According  to  the 
program  staff,  the  NAE  advised  them  that  the  AoA  could  be  approved  by  the 
PEO,  but  they  could  not  provide  supporting  documentation. 

Acquisition  Program  Baseline.  The  NAE  approved  acquisition  program 
baselines  for  12  of  the  13  programs.  The  acquisition  program  baseline  is  prepared 
by  the  program  manager  and  approved  by  the  NAE  prior  to  milestone  approval. 
The  acquisition  program  baseline  prescribes  the  key  cost,  schedule,  and 
performance  constraints  that  must  be  achieved  by  the  program  before  the  next 
milestone  decision  review  in  the  acquisition  process.  The  constraints  are 
documented  in  the  objective  value  and  threshold  value  for  each  parameter.  The 
program  manager  manages  the  program  to  the  objective  value  of  each  parameter. 
Thresholds  represent  the  acceptable  limits  to  the  parameter  values  that  still 
provide  the  needed  capability.  The  failure  to  attain  program  thresholds  may 
degrade  system  performance,  delay  the  program,  or  make  the  program  too  costly. 
Program  managers  are  responsible  for  executing  programs  in  accordance  with 
approved  cost,  schedule,  and  performance  thresholds  set  in  the  acquisition 
program  baseline. 


Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal.  The 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal  program’s 
acquisition  program  baseline  was  not  approved  until  6  weeks  after  its  acquisition 
decision  memorandum  for  Milestone  B.  ’  The  NAE  decided  to  withhold  approval 
of  the  acquisition  program  baseline  until  contract  award,  which  had  been  delayed 
to  adjust  contractor  incentives  and  was  dependent  upon  the  Milestone  B  decision. 
The  acquisition  program  baseline  dates  would  be  affected  by  the  delay  in  contract 
award  and  the  NAE  decided  to  defer  final  signature  until  firm  dates  could  be 
determined  and  the  acquisition  program  baseline  updated.  The  NAE  approved  the 
acquisition  decision  memorandum  for  Milestone  B  on  October  21,  2003.  The 
contracts  were  awarded  on  October  29,  2003.  The  acquisition  program  baseline 
was  then  updated  to  reflect  the  new  dates  and  the  NAE  approved  it  on 
December  3,  2003.  Regardless  of  the  change  in  the  contract,  the  acquisition 
program  baseline  should  still  have  been  approved  before  the  acquisition  decision 
memorandum  as  required  by  DoD  Instruction  5000.2.  The  NAE  would  be  unable 
to  assess  the  cost,  schedule,  and  performance  parameters  of  the  program  without 
an  accurately  defined  acquisition  program  baseline. 


Acquisition  Strategies.  The  NAE  approved  four  acquisition  decision 
memorandums  even  though  two  programs  did  not  have  a  properly  approved 
acquisition  strategy  and  two  programs  did  not  have  a  complete  acquisition 
strategy.  An  acquisition  strategy  is  a  business  and  technical  management 
approach  designed  to  achieve  program  objectives  within  the  imposed  resource 
constraints.  It  is  the  framework  for  planning,  directing,  contracting  for,  and 
managing  a  program.  It  provides  a  master  schedule  for  research,  development, 
test,  production,  fielding,  modification,  postproduction  management,  and  other 


3  At  Milestone  B,  a  recommendation  is  made  and  approval  sought  for  the  program  to  leave  the  technology 
development  phase  and  continue  to  the  system  development  and  demonstration  phase  of  the  acquisition 
lifecycle. 
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activities  essential  for  program  success.  The  acquisition  strategy  is  required  by 
DoD  Instruction  5000.2  for  each  program  and  is  prepared  by  the  program 
manager  and  approved  by  the  NAE.  If  an  acquisition  strategy  was  not  ready  for 
approval  at  the  time  the  acquisition  decision  memorandum  was  approved,  the 
program  may  not  have  been  ready  to  proceed  to  the  next  acquisition  phase  and  the 
NAE  would  be  unable  to  determine  the  program’s  complete  technical  and 
management  approach. 

AN/APG-79  Active  Electronically  Scanned  Array  Radar  System.  The 

NAE  approved  the  AN/APG-79  Active  Electronically  Scanned  Array  Radar 
System’s  acquisition  strategy  22  days  after  approving  the  program’s  acquisition 
decision  memorandum  for  Milestone  C.4  During  the  Milestone  C  meeting,  the 
ASN(RD&A)  made  changes  to  the  program  involving  a  fourth  LRIP.  As  a  result 
of  these  changes,  the  acquisition  strategy  needed  to  be  revised  and  resubmitted  to 
ASN(RD&A)  for  signature,  leading  to  the  22-day  delay. 

AN/SPY-1D(V)  Radar  Upgrade.  The  AN/SPY-1D(V)  system’s  program 
manager  did  not  develop  an  acquisition  strategy;  instead  program  officials 
provided  an  acquisition  plan  for  the  program.  We  reviewed  the  plan  to  determine 
whether  it  contained  the  same  information  required  by  DoD  Instruction  5000.2  for 
acquisition  strategy.  We  determined  that  the  acquisition  plan  contained  the 
majority  of  the  information  required  of  the  acquisition  strategy;  however,  the 
acquisition  plan  does  not  contain  justification  of  low-rate  initial  production 
quantities  exceeding  10  percent  and  was  not  approved  by  the  NAE. 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  Upgrade.  The 

Mode  5  acquisition  strategy  was  approved  6  months  after  the  NAE  approved  the 
acquisition  decision  memorandum  for  Milestone  B.  The  Deputy  Assistant 
Secretary  of  the  Navy  action  officer  stated  that  during  the  milestone  decision 
briefing,  the  NAE  requested  several  changes  be  made  to  the  program’s  acquisition 
strategy  before  he  would  approve  it.  The  NAE  approved  proceeding  to  the  next 
acquisition  phase  because  he  did  not  want  to  delay  the  Milestone  B  decision.  It 
took  several  months  for  the  NAE-requested  changes  to  be  re-staffed  and  the 
acquisition  strategy  finally  approved.  However,  program  office  personnel  stated 
that  the  delay  in  approval  of  the  acquisition  strategy  was  due  to  staff  schedules  at 
the  NAE  level  and  that  the  acquisition  strategy  never  changed  after  it  was 
submitted  at  the  milestone  review.  After  examining  the  acquisition  strategy  we 
determined  that  the  NAE  signature  was  the  only  one  dated  after  the  acquisition 
decision  memorandum  was  approved. 

Surface  Electronic  Warfare  Improvement  Program.  The  Surface 
Electronic  Warfare  Improvement  Program  (SEWIP)  had  an  approved  acquisition 
strategy  for  Block  1  A;  however,  the  strategy  was  incomplete  because  it  did  not 
include  a  section  on  human  systems  integration  or  its  components  as  required  by 
DoD  Instruction  5000.2.  DoD  Instruction  5000.2  requires  the  program  manager 
to  have  a  comprehensive  plan  for  human  systems  integration  in  place  early  in  the 
acquisition  process  to  optimize  total  system  performance,  minimize  total 


4  At  Milestone  C,  a  recommendation  is  made  and  approval  sought  for  the  program  to  leave  the  system 
development  and  demonstration  phase  and  continue  into  the  production  and  deployment  phase  of  the 
acquisition  life  cycle.  This  milestone  also  authorizes  low-rate  initial  production  for  the  program. 


9 


ownership  costs,  and  ensure  that  the  system  is  built  to  accommodate  the 
characteristics  of  the  user  population  that  will  operate,  maintain,  and  support  the 
system.  A  SEWIP  program  official  provided  a  separate  human  systems 
integration  plan  for  the  program;  however,  it  was  developed  after  both  spirals  of 
Block  1 A  were  approved  for  Milestone  C.  The  Deputy  Assistant  Secretary  of  the 
Navy  action  officer  stated  that  only  spiral  2  of  Block  1A  required  human  systems 
integration  planning.  The  action  officer  stated  that  instead  of  including  the  plan 
in  the  acquisition  strategy  for  Block  1A,  the  program  staff  created  a  separate  plan 
as  an  independent  document  for  the  program  and  that  the  plan  was  completed  or 
near  completion  when  the  NAE  approved  the  Milestone  C  acquisition  decision 
memorandum  for  spiral  2  of  Block  1A.  Because  human  systems  integration  was 
not  developed  early  in  the  acquisition  process,  the  program’s  system  performance 
and  total  ownership  costs  may  be  affected,  along  with  optimized  total  system 
performance. 

Information  Support  Plan.  Two  programs  did  not  have  an  infonnation  support 
plan  when  the  NAE  approved  their  acquisition  decision  memorandums.  The 
information  support  plan  (formerly  known  as  the  command,  control, 
communications,  computers,  and  intelligence  support  plan)  identifies  and 
documents  information  needs,  infrastructure  support,  information  technology,  and 
National  Security  Systems  interface  requirements  and  dependencies.  The 
information  support  plan  focuses  on  net-centricity,  interoperability, 
supportability,  and  sufficiency  concerns.  Without  the  infonnation  support  plan, 
the  NAE  did  not  have  all  the  required  information  to  determine  the  program’s 
information  needs,  infrastructure  support,  information  technology,  and  interface 
requirements  and  dependencies. 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  Upgrade.  The 

Mode  5  program  did  not  have  an  information  support  plan  when  the  NAE 
approved  its  acquisition  decision  memorandum  for  the  system  development  and 
demonstration  phase  on  August  7,  2003.  The  program  manager  did  not  complete 
or  obtain  approval  for  an  infonnation  support  plan  because  he  did  not  believe  that 
the  document  was  a  requirement  even  though  the  program’s  operational 
requirements  document  (ORD)  identifies  system  interoperability  requirements  as 
a  key  perfonnance  parameter.  On  June  28,  2006,  the  Deputy  Assistant  Secretary 
of  the  Navy  assigned  to  the  Mode  5  program  approved  a  waiver  for  the  required 
information  support  plan.  However,  the  determination  that  the  program  did  not 
require  an  information  support  plan  should  have  been  made  before  the  milestone 
decision.  The  NAE  should  have  required  the  program  to  either  develop  an 
information  support  plan  or  obtain  a  waiver  prior  to  approving  the  next  milestone. 

AV-8B  Harrier  II  Open  Systems  Core  Avionics  Requirements. 

Program  officials  for  AV-8B  Harrier  II  Open  Systems  Core  Avionics 
Requirements  (OSCAR)  obtained  a  command,  control,  communications, 
computers,  and  intelligence  support  plan  waiver  from  Deputy  Assistant  Secretary 
of  the  Navy  Command,  Control,  Communications,  Computers,  and  Intelligence 
and  Space;  however,  the  waiver  was  not  properly  approved.  DoD  Regulation 
5000.2-R,  “Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  and 
Major  Automated  Information  System  Acquisition  Programs,”  June  10,  2001, 
states  that  an  NAE,  with  the  advice  from  the  appropriate  Chief  Infonnation 
Officer,  may  waive  command,  control,  communications,  computers,  and 
intelligence  support  plan  preparation  if  the  Requirements  Authority  has 
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previously  waived  the  requirement  for  an  interoperability  key  performance 
parameter  in  the  ORD.  The  Deputy  Assistant  Secretary  of  the  Navy  Command, 
Control,  Communications,  Computers,  and  Intelligence  and  Space  approved  a 
waiver  for  the  program’s  command,  control,  communications,  computers,  and 
intelligence  support  plan.  The  waiver  was  granted  because  the  program  did  not 
have  an  interoperability  key  performance  parameter  in  the  AV-8B  Radar  and 
Night  Attack  ORDs.  OSCAR  program  officials  currently  consider  these  two 
ORDs  as  their  operational  requirements  documents.  However,  as  noted  in  the 
ORD  section  of  this  finding,  we  detennined  that  OSCAR  should  have  developed  a 
program-specific  ORD.  In  addition,  the  waiver  was  not  approved  by  the  NAE  as 
required  by  DoD  Regulation  5000. 2-R.  As  a  result,  the  reason  for  approving  and 
the  approval  authority  for  the  waiver  were  invalid;  the  program  should  have 
developed  a  command,  control,  communications,  computers,  and  intelligence 
support  plan. 

Mission  Needs  Statement.  Nine  of  the  13  Navy  programs  we  reviewed  had  a 
mission  needs  statement.  The  mission  needs  statement  documents  the  required 
operational  capabilities  of  a  system.  The  mission  needs  statement  must  be 
prepared  in  accordance  with  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
3170.01,  “Requirements  Generation  System,”  August  10,  1999,  and  validated 
prior  to  the  beginning  of  the  program.  The  mission  needs  statement  is  a  legacy 
document  and  has  been  replaced  by  the  initial  capabilities  document. 

MH-60R  Multi-Mission  Helicopter.  Program  officials  for  MH-60R 
Multi-Mission  Helicopter  did  not  believe  that  there  was  a  mission  needs  statement 
developed  for  the  program.  Instead,  the  need  for  the  system  was  documented  in 
an  ORD.  The  ORD,  dated  October  28,  2004,  stated  that  “.  .  .  there  is  a  mission 
need  for  an  interdependent  ship/air  weapons  system  to  extend  and  increase 
Surface  Warfare,  Undersea  Warfare,  sea  control  and  power  projection  capabilities 
of  surface  forces  in  the  conduct  of  offensive  and  defensive  missions  that  support 
national  interests.” 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal.  The 

need  for  Navy  Advanced  Extremely  High  Frequency  Multiband  Tenninal  is 
described  in  the  following  mission  needs  statements: 

•  April  23,  1996,  Follow-on  Military  Satellite  Communications 
Mission  Needs  Statement  (United  States  Space  Command 
Mission  Needs  Statement  002-94); 

•  April  23,  1996,  Follow-on  Military  Satellite  Communications 
Mission  Needs  Statement  (United  States  Space  Command 
Mission  Needs  Statement  002-95);  and 

•  August  3,  1998,  Global  Broadcast  Service  Mission  Needs 
Statement. 

On  a  number  of  occasions,  we  tried  to  obtain  a  copy  of  the  mission  needs 
statements  but  program  officials  were  not  responsive.  Without  reviewing  these 
documents,  we  cannot  verify  whether  the  mission  needs  statements  were 
developed  or  that  they  support  the  need  for  the  Navy  Advanced  Extremely  High 
Frequency  Multiband  Terminal. 
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Open  Systems  Core  Avionics  Requirements.  The  OSCAR  program 
office  did  not  develop  a  mission  needs  statement.  Instead,  the  program  officials 
stated  that  the  OSCAR  program  was  started  as  a  demonstration  program 
designated  by  the  Secretary  of  Defense. 

Operational  Requirements  Document.  Twelve  of  the  13  Navy  programs  we 
reviewed  had  an  ORD.  The  ORD  is  a  legacy  document  that  contains  operational 
performance  requirements  and  parameters  for  the  proposed  concept  or  system. 
DoD  Regulation  5000. 2-R,  March  15,  1996,  requires  that  an  ORD  be  developed 
for  all  acquisition  programs.  The  program  sponsor  develops  the  ORD,  the  Chief 
of  Naval  Operations  or  Commandant  of  the  Marine  Corps  approves  it,  and  the 
Joint  Requirements  Oversight  Council  validates  it. 

Open  Systems  Core  Avionics  Requirements.  The  program  sponsor  did 
not  develop  an  ORD  for  the  OSCAR  program;  however,  the  NAE  approved  the 
program  to  proceed  through  all  milestone  decisions,  including  full-rate 
production,  without  requiring  the  program  manager  to  develop  an  ORD.  As  a 
result,  we  were  unable  to  determine  the  warfighter  requirement  for  the  OSCAR 
acquisition  program  or  identify  the  capability  gap  that  the  OSCAR  acquisition 
program  was  intended  to  fill.  In  addition,  without  an  approved  ORD,  the  NAE 
was  unable  to  determine  the  operational  performance  requirements  and 
parameters  for  the  proposed  system.  As  of  August  2004,  the  program  officials  did 
not  intend  to  issue  an  OSCAR-specific  ORD. 

Systems  Engineering  Plan.  Two  programs  did  not  have  a  properly  approved 
systems  engineering  plan  prior  to  milestone  approval.  The  systems  engineering 
plan  describes  the  program’s  overall  approach,  including  processes,  resources, 
metrics,  and  applicable  perfonnance  incentives.  The  plan  also  details  the  timing, 
conduct,  and  success  criteria  of  technical  reviews.  Systems  engineering  focuses 
on  defining  user  needs  and  required  functionality  early  in  the  development  cycle, 
documenting  requirements,  then  proceeding  with  design  synthesis  and  system 
validation  to  achieve  the  total  capability. 

AN/AES-1  Airborne  Laser  Mine  Detection  System.  Program  officials 
for  the  AN/AES-1  Airborne  Laser  Mine  Detection  System  provided  the 
program’s  system  engineering  management  plan  instead  of  a  systems  engineering 
plan.  The  system  engineering  management  plan  contained  the  same  elements  as 
required  of  the  systems  engineering  plan.  However,  the  system  engineering 
management  plan  was  not  approved  by  the  NAE  as  required  by  the  February  20, 
2004,  USD(AT&L)  memorandum.  The  memorandum  was  issued  approximately 
1  year  and  4  months  prior  to  the  program’s  Milestone  C  approval,  which  should 
have  provided  the  program  ample  time  to  get  NAE  approval  of  the  document.  As 
a  result,  the  program  did  not  have  a  valid  systems  engineering  plan  when  the  NAE 
issued  an  acquisition  decision  memorandum  for  Milestone  C. 

Surface  Electronic  Warfare  Improvement  Program.  The  SEWIP 
program  provided  the  SEWIP  system  engineering  management  plan  instead  of  a 
systems  engineering  plan.  The  system  engineering  management  plan  is  a  legacy 
document  that  has  been  replaced  by  the  systems  engineering  plan.  Both 
documents  contain  the  same  elements.  However,  SEWIP  system  engineering 
management  plan  was  not  approved  by  the  NAE  as  required  by  the  USD(AT&L) 
memorandum,  “Policy  for  Systems  Engineering  in  DoD,”  February  20,  2004. 


12 


The  memorandum  requires  that  all  programs  develop  a  systems  engineering  plan 
for  milestone  decision  authority  approval  in  conjunction  with  each  milestone 
review.  The  memorandum  was  issued  approximately  1  year  before  Block  1A 
spiral  1  was  approved  for  Milestone  C  and  1  year  and  9  months  before  Block  1A 
spiral  2  was  approved  for  Milestone  C,  which  should  have  provided  the  program 
office  ample  time  to  get  NAE  approval  of  the  document.  However,  the  program 
did  not  have  an  approved  systems  engineering  plan  when  the  NAE  issued  the 
acquisition  decision  memorandum  for  Milestone  C. 

Test  and  Evaluation  Master  Plan.  The  NAE  approved  acquisition  decision 
memorandums  for  two  programs  that  had  test  and  evaluation  master  plans  that  the 
Director,  Operational  Test  and  Evaluation  had  not  yet  approved.  The  test  and 
evaluation  master  plan  documents  the  overall  structure  and  objectives  of  the  test 
and  evaluation  program.  The  program  manager  submits  it  at  Milestone  B  and  C, 
and  at  the  full-rate  production  decision  review.  The  Director,  Operational  Test 
and  Evaluation  must  approve  the  plan  for  AC  AT  IC  programs  and  for  programs 
on  the  Office  of  the  Secretary  of  Defense  Test  and  Evaluation  oversight  list.  The 
NAE  approves  the  test  and  evaluation  master  plan  for  ACAT  II  programs.  The 
test  and  evaluation  master  plan  describes  planned  developmental,  operational,  and 
live-fire  testing.  It  includes  measures  to  evaluate  the  performance  of  the  system 
during  these  test  periods,  an  integrated  test  schedule,  and  the  resources 
requirements  to  accomplish  the  planned  testing. 

MH-60R  Multi-Mission  Helicopter.  Even  though  the  Director, 
Operational  Test  and  Evaluation  had  not  approved  the  MH-60R  test  and 
evaluation  master  plan  Revision  C,  the  NAE  approved  full-rate  production  on 
March  31,  2006.  Revision  C  of  the  test  and  evaluation  master  plan  addresses 
follow-on  operational  test  and  evaluation  activities  for  the  MH-60R  upgrades  and 
enhancements  in  the  form  of  planned  product  improvements.  The  program 
manager  must  submit  the  approved  test  and  evaluation  master  plan  as  part  of  the 
full-rate  production  decision  review.  Commander,  Operational  Test  and 
Evaluation  Force  conducts  follow-on  operational  test  and  evaluation  during  the 
operations  and  support  phase.  Therefore,  the  test  and  evaluation  master  plan 
Revision  C  should  have  been  approved  prior  to  full-rate  production  approval. 
Because  the  Director,  Operational  Test  and  Evaluation  had  not  approved  the  test 
and  evaluation  master  plan  Revision  C,  the  NAE  could  not  be  assured  that 
planned  follow-on  operational  test  and  evaluation  were  appropriate  or  adequate. 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal.  The 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal  program’s  test 
and  evaluation  master  plan  was  not  approved  by  Director,  Operational  Test  and 
Evaluation  until  almost  2  years  after  the  NAE  approved  the  acquisition  decision 
memorandum  for  Milestone  B.  The  program  was  approved  for  Milestone  B 
because  the  NAE  decided  that  a  Navy-approved  test  and  evaluation  master  plan 
was  adequate  even  though  the  program  was  listed  on  the  Director,  Operational 
Test  and  Evaluation  oversight  list.  After  the  milestone  review,  the  program 
manager  continued  to  experience  delays  in  getting  the  test  and  evaluation  master 
plan  approved  by  the  Director,  Operational  Test  and  Evaluation.  Program 
officials  revised  the  plan  several  times  as  a  result  of  unanticipated  delays  in 
finalizing  the  source  selection  process  for  the  Navy  Advanced  Extremely  High 
Frequency  Multiband  Terminal.  The  contract  for  the  system  development  and 
demonstration  phase  was  awarded  on  October  29,  2003,  eight  days  after  the  NAE 
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approved  the  acquisition  decision  memorandum.  The  Director,  Operational  Test 
and  Evaluation  approved  the  test  and  evaluation  master  plan  on  September  8, 
2005.  Because  the  Director,  Operational  Test  and  Evaluation  had  not  approved 
the  plan  before  the  NAE  approved  the  acquisition  decision  memorandum,  the 
NAE  could  not  be  assured  that  the  overall  structure  and  objectives  of  the  test  and 
evaluation  program  for  the  Navy  Advanced  Extremely  High  Frequency  Multiband 
Terminal  were  appropriate  or  adequate. 


Milestone  Documentation  Agreements 


The  programs  were  allowed  to  proceed  without  approval  of  required  acquisition 
documentation  and  the  decision  was  not  formally  noted  in  a  waiver,  tailored 
acquisition  agreement,  or  acquisition  decision  memorandum.  After  meeting  with 
Deputy  Assistant  Secretary  of  the  Navy  action  officers,  we  learned  that  when  a 
required  document  is  not  provided  or  properly  approved  at  a  milestone  decision, 
as  required  by  DoD  Instruction  5000.2,  the  NAE  will  typically  approve  a  waiver 
for  the  document  or  add  a  provision  in  the  acquisition  decision  memorandum  that 
requires  the  program  manager  to  complete  and  submit  it  for  approval  before  the 
program  can  advance  further  through  the  acquisition  process.  In  addition, 
SECNAV  Instruction  5000.2C  states  that  the  NAE  may  tailor  documentation  and 
milestone  requirements.  The  NAE  is  required  to  approve  the  streamlined,  tailored 
documentation  approach  during  program  initiation  and  prior  to  all  other 
milestones.  Only  2  out  of  the  10  programs  with  missing  or  improperly  approved 
documents  had  formal  tailoring  agreements.  However,  even  those  programs  were 
missing  or  had  improperly  approved  documents.  In  addition,  we  were  unable  to 
locate  properly  approved  waivers  or  provisions  in  the  acquisition  decision 
memorandums  for  the  programs  with  missing  or  improperly  approved  documents. 


Results  of  Milestone  Review  Process 


When  asked  why  the  programs  were  allowed  to  proceed  to  the  next  milestone 
without  key  documents  or  approval  of  key  documents,  the  responses  varied  from 
program  office  staff  and  representatives  from  the  Deputy  Assistant  Secretary  of 
the  Navy  level.  In  one  instance,  program  officials  could  not  offer  an  explanation. 
In  general,  key  acquisition  documents  were  either  not  provided  for  milestone 
reviews  or  were  not  properly  approved  at  milestone  reviews  because  DoD  and 
Navy  policies  outlining  the  acquisition  process  and  documentation  requirements 
were  misinterpreted  at  the  program  level  or  not  followed  by  the  NAE.  In 
addition,  it  was  unclear  why  the  NAE  allowed  programs  to  proceed  through 
milestones  without  proper  documentation  because  formal  tailored  documentation 
agreements  did  not  exist,  waivers  for  missing  documents  were  not  done  or  were 
not  properly  approved,  and  documentation  decisions  associated  with  each 
milestone  review  were  not  being  documented  in  acquisition  decision 
memorandums.  As  a  result,  the  NAE  did  not  have  all  the  necessary  information 
required  to  make  fully  informed  milestone  decisions. 
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Updating  Program  Documentation  Between  Milestone  Reviews 


Of  the  13  programs  reviewed,  7  had  identified  deviations  in  performance, 
schedule,  or  cost  thresholds  between  milestone  reviews.  We  found  that  one  of  the 
seven  program  managers  had  not  submitted  a  formal  deviation  report  as  required 
by  SECNAV  Instruction  5000. 2C.  For  three  of  the  seven  programs,  either  the 
program  manager  did  not  provide  the  NAE  with  updated  acquisition  program 
baselines  between  milestone  decision  reviews  or  the  NAE  did  not  approve  the 
updated  acquisition  program  baselines  in  a  timely  manner. 

Acquisition  Program  Baselines.  The  acquisition  program  baseline  prescribes 
the  key  cost,  schedule,  and  perfonnance  constraints  that  must  be  achieved  by  the 
program  before  the  next  milestone  decision  review  in  the  acquisition  process. 

The  constraints  are  documented  in  the  objective  and  threshold  values  for  each 
parameter,  and  the  program  manager  manages  the  program  to  the  objective  value 
of  each  parameter.  Thresholds  represent  the  acceptable  limits  to  the  parameter 
values  that  still  provide  the  needed  capability.  The  failure  to  attain  program 
thresholds  may  degrade  system  performance,  delay  the  program,  or  make  the 
program  too  costly.  The  failure  to  attain  program  thresholds  places  the  overall 
affordability  of  the  program  and  the  capability  provided  by  the  system  into 
question.  The  program  manager  revises  the  acquisition  program  baseline  for  each 
milestone  review.  The  program  manager  also  revises  the  acquisition  program 
baseline  if  the  program  is  restructured  or  has  an  unrecoverable  deviation. 

Program  Deviations.  The  Defense  Acquisition  Guidebook  states  that  a  program 
deviation  occurs 

when  the  program  manager  believes  that  the  current  estimate  for  the 
program  indicates  that  a  performance,  schedule,  or  cost  threshold  value 
will  not  be  achieved.  Within  30  days  of  the  program  deviation,  the 
program  manager  should  notify  the  NAE  of  the  reason  for  the  program 
deviation  and  the  actions  that  will  bring  the  program  back  within  the 
baseline  parameters.  Within  90  days  of  the  program  deviation,  one  of 
the  following  should  have  occurred:  the  program  is  back  within 
acquisition  program  baseline  parameters;  a  new  acquisition  program 
baseline,  changing  only  those  parameters  that  were  breached,  has  been 
approved;  or  an  Overarching  Integrated  Product  Team  level  program 
review  has  been  conducted  to  review  the  program  manager’s  proposed 
baseline  revisions  and  make  recommendations  to  the  milestone 
decision  authority  for  preparing,  submitting,  and  updating  required 
program  documentation. 

SECNAV  Instruction  5000.2C  states  that  the  program  manager  must  provide  a 
deviation  report  to  the  NAE  immediately  upon  program  deviation. 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal.  The 

program  manager  issued  four  deviation  reports  to  the  ASN(RD&A)  for  the  Navy 
Advanced  Extremely  High  Frequency  Multiband  Terminal  program.  The  first 
report  was  issued  on  June  29,  2004,  and  stated  that  an  updated  acquisition 
program  baseline  would  be  submitted  within  90  days.  A  second  deviation  report, 
issued  on  October  19,  2004,  updated  the  initial  deviation  report  and  stated  that  the 
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revised  acquisition  program  baseline  would  be  submitted  within  270  days.  A 
third  deviation  report,  issued  on  February  22,  2005,  updated  the  initial  deviation 
report  and  stated  that  a  revised  acquisition  program  baseline  would  be  submitted 
in  June  2005.  A  fourth  deviation  report,  issued  on  May  18,  2005,  updated  the 
initial  deviation  report  and  reiterated  that  a  revised  acquisition  program  baseline 
would  be  submitted  in  June  2005.  The  Defense  Acquisition  Guidebook 
recommends  that  an  acquisition  program  baseline  be  updated  and  approved  within 
90  days  of  a  program  deviation  or  that  an  Overarching  Integrated  Product  Team 
conduct  a  baseline  review.  The  program’s  Deputy  Assistant  Secretary  of  the 
Navy  action  officer  stated  that  after  a  June  9,  2006,  briefing  to  the  NAE  by  the 
program  manager,  the  NAE  approved  the  acquisition  program  baseline.  As  of 
October  2006,  the  acquisition  program  baseline  was  not  yet  approved.  As  a 
result,  we  were  unable  to  verily  that  an  updated  acquisition  program  baseline  was 
developed  and  approved  for  the  program.  Even  if  the  program  had  an  updated 
and  approved  acquisition  program  baseline,  the  program  was  still  outside 
acquisition  program  baseline  parameters  for  2  years.  In  addition,  an  Overarching 
Integrated  Product  Team  still  has  not  properly  conducted  a  baseline  review. 

Open  Systems  Core  Avionics  Requirements.  The  OSCAR  program 
manager  issued  a  program  deviation  report  on  April  7,  2003.  The  program 
manager  signed  an  updated  acquisition  program  baseline  on  July  14,  2003;  the 
PEO  concurred  July  16,  2003;  and  the  NAE  approved  it  on  November  24,  2003, 
130  days  after  the  PEO  concurred.  Based  on  these  dates,  the  program  manager 
did  not  have  an  updated  and  approved  acquisition  program  baseline  within 
90  days  as  specified  in  the  Defense  Acquisition  Guidebook. 

AN/AES-1  Airborne  Laser  Mine  Detection  System.  In  April  2004,  the 
AN/AES-1  Airborne  Laser  Mine  Detection  System  program  deviated  from  the 
approved  acquisition  program  baseline.  Although  the  program  manager 
developed  a  deviation  report  memorandum,  it  was  never  forwarded  to  the  NAE. 
Instead,  the  program  manager  infonned  the  NAE  of  the  deviation  through  a  series 
of  status  reviews  that  took  place  from  June  2004  through  March  2005.  Those 
reviews  identified  the  acquisition  program  baseline  deviation  and  proposed 
strategy  changes  that  would  affect  the  program’s  cost  and  schedule.  The  strategy 
changes  were  identical  to  those  proposed  in  the  deviation  report  memorandum 
that  the  program  manager  did  not  submit  to  the  NAE.  Instead  of  routing  the 
deviation  report  memorandum  to  the  NAE,  the  program  manager  focused  on 
routing  and  obtaining  signatures  on  the  updated  acquisition  program  baseline 
because  he  felt  the  briefings  appropriately  notified  the  NAE  of  the  program’s 
deviations.  However,  the  June  2004  presentation  recommended  the  development 
of  a  deviation  memorandum,  and  a  September  2004  presentation  updated  the 
status  of  the  deviation  memorandum  as  being  signed  by  the  program  manager  and 
en  route  to  the  PEO.  SECNAV  Instruction  5000.2C  allows  the  program  manager 
to  choose  the  deviation  report’s  presentation  medium.  Every  program  deviation 
report  we  reviewed  was  provided  in  the  form  a  memorandum.  Had  the  program 
manager  intended  the  presentations  to  serve  as  the  deviation  report,  the  program 
manager  would  not  have  developed  a  deviation  report  memorandum.  A  deviation 
report  should  have  been  provided  to  the  NAE  immediately  upon  program 
deviation  as  required  by  DoD  Instruction  5000.2  and  SECNAV  Instruction 
5000.2C.  An  updated  acquisition  program  baseline  was  approved  June  14,  2005, 
when  the  program  had  a  Milestone  C  decision,  approximately  1  year  after  the 
NAE  was  first  informed  of  the  deviation  in  program  briefings. 
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Tracking  Program  Deviations  in  Dashboard 


The  NAE  does  not  track  program  deviations  or  their  causes,  effects,  and 
mitigations  in  the  Navy  Dashboard  database.  We  reviewed  the  Navy  Dashboard 
database  to  determine  the  status  of  the  deviation  reports  for  each  program; 
however,  we  discovered  that  program  officials  do  not  track  deviation  reports  and 
mitigation  strategies  in  the  Navy  Dashboard  database.  In  the  Dashboard  database, 
anytime  the  program  manager’s  current  estimate  exceeds  the  threshold  value,  the 
cause  of  the  breach  is  required  to  be  entered  in  a  field  contained  in  the  database. 
This  does  not  replace  the  requirement  for  the  program  manager  to  immediately 
notify  the  NAE  of  the  breach  to  the  acquisition  program  baseline.  Other  than  the 
input  in  the  Dashboard  database,  the  NAE  had  no  way  to  track  the  deviation 
report  and  its  mitigation  strategy  for  bringing  the  program  back  within  acquisition 
program  baseline  parameters. 


Monitoring  Programs  Between  Milestones 


The  NAE  was  unable  to  determine  whether  the  programs  met  cost,  schedule,  and 
performance  constraints  between  milestone  reviews  because  the  objective  and 
threshold  parameters  within  the  acquisition  program  baselines  were  no  longer 
current  for  the  programs.  Without  an  updated  acquisition  program  baseline,  the 
program  manager  had  no  cost,  schedule,  and  performance  constraints  to  manage 
the  program  between  milestone  reviews. 


Management  Comments  on  the  Finding  and  Audit  Response 


Summaries  of  management  comments  on  the  finding  and  our  audit  response  are  in 
Appendix  H. 


Recommendations,  Management  Comments,  and  Audit 
Response 


Revised  and  Deleted  Recommendations.  The  Deputy  Assistant  Secretary  of  the 
Navy  (Management  and  Budget)  provided  management  comments  on  behalf  of 
the  ASN(RD&A).  As  a  result  of  management  comments,  we  revised  draft 
Recommendation  A. 3.  and  Recommendation  A. 5.  to  clarify  the  intent  of  the 
recommendations.  Additionally,  we  deleted  Recommendation  A.l.e.  because  it 
duplicated  Recommendation  A.l.e. 

A.l.  We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition): 

a.  Comply  with  Under  Secretary  of  Defense  Acquisition,  Technology, 
and  Logistics  memorandum,  “Policy  for  Systems  Engineering  in  DoD,” 
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February  20,  2004;  DoD  Instruction  5000.2;  and  Secretary  of  the  Navy 
Instruction  5000.2C  when  reviewing  and  approving  systems  engineering 
plans  as  part  of  a  program’s  milestone  review. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  in  June  2005,  the  ASN(RD&A)  issued  Department  of  the  Navy 
memorandum,  “Policy  for  Department  of  the  Navy  Systems  Engineering  Plan 
Review  and  Approval,”  to  implement  the  requirements  of  the  USD(AT&L) 
memorandum.  In  April  2006,  the  ASN(RD&A)  consolidated  responsibility  for 
systems  engineering  plan  development  and  approval  with  memorandum,  “Revised 
Policy  for  Department  of  Navy  Systems  Engineering  Plan  Review  and  Approval.” 
In  accordance  with  the  memorandum,  the  ASN(RD&A)  Chief  Engineer  has  been 
delegated  the  responsibility  for  approval  of  all  ACAT  IC  and  II  systems 
engineering  plans.  The  responsibilities  of  the  PEOs  and  Systems  Commanders  to 
ensure  timely  development  and  implementation  of  quality  systems  engineering 
plans  were  emphasized,  and  the  Chief  Engineer  has  the  primary  responsibility  for 
their  approval.  In  addition,  the  SECNAV  Instruction  5000.2D  has  been  updated 
accordingly. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 

b.  Review,  sign,  date,  and  maintain  decision  documents  in  the 
program  files  as  a  result  of  milestone  reviews  in  the  acquisition  decision 
memorandum,  waiver,  or  tailored  documentation  agreement. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  ASN(RD&A)  will  formally  document  decisions  regarding  tailoring 
or  waiver  of  documentation  in  future  acquisition  decision  memorandums, 
tailoring  agreements,  or  waiver  approvals.  The  Deputy  Assistant  Secretary  of  the 
Navy  (Management  and  Budget)  also  stated  that  these  documents  will  be 
maintained  in  the  program  files  for  future  reference. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 

c.  Comply  with  the  Defense  Acquisition  Guidebook  recommendations 
to  update  and  approve  acquisition  program  baselines  for  programs  between 
milestone  decision  reviews  within  90  days. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  partially  concurred. 
The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that 
although  the  milestone  decision  authority’s  approval  of  the  revised  acquisition 
program  baseline  is  desired  within  90  days  of  the  deviation,  it  is  not  always 
feasible.  He  further  stated  that  given  the  number  of  ACAT  programs,  the 
volatility  of  these  efforts,  the  normal  time  required  to  develop  an  independent  cost 
estimate,  and  all  other  administrative  issues  associated  with  acquisition  program 
baseline  revisions,  approval  by  the  milestone  decision  authority  is  not  always 
feasible  within  90  days.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  stated  that  the  Defense  Acquisition  Guidebook  is 
considered  guidance  and  is  not  regulatory  or  required.  He  stated  that  the 
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Department  of  the  Navy  strives  to  obtain  approval  of  revised  acquisition  program 
baselines  as  expediently  as  possible. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation.  We 
understand  that  it  is  not  always  feasible  to  have  the  milestone  decision  authority’s 
approval  within  90  days.  We  also  understand  that  the  Defense  Acquisition 
Guidebook  is  considered  guidance  and  not  regulatory  or  required.  However,  the 
Defense  Acquisition  Guidebook  does  provide  alternative  actions  if  the  milestone 
decision  authority  is  unable  to  approve  the  acquisition  program  baseline  within  90 
days.  The  Defense  Acquisition  Guidebook  states  that  within  90  days  of  the 
program  deviation,  one  of  the  following  should  have  occurred:  the  program  is 
back  within  acquisition  program  baseline  parameters;  a  new  acquisition  program 
baseline,  changing  only  those  parameters  that  were  breached,  has  been  approved; 
or  an  Overarching  Integrated  Product  Team  level  program  review  has  been 
conducted  to  review  the  program  manger’s  proposed  baseline  revisions  and  make 
recommendations  to  the  milestone  decision  authority. 

d.  Track  deviation  reports  in  the  Navy  Dashboard  database  to 
include  the  date,  cause,  effect,  and  mitigation. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  ASN(RD&A)  will  make  the  necessary  changes  to  the  Navy 
Dashboard  database  and  start  tracking  deviation  reports  and  baseline  breaches.  In 
a  follow-up  conversation  regarding  management  comments,  an  ASN(RD&A) 
representative  indicated  that  Dashboard  should  be  modified  to  track  deviation 
reports  by  the  end  of  June  2007. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 

A.2.  We  recommend  that  program  managers  maintain  approved 
documentation  for  the  life  of  the  acquisition  program. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  program  offices  are  expected  to  maintain  appropriate  repositories  of 
program  documentation  for  the  life  cycle  of  the  acquisition  program.  The  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  also  stated  that  the 
ASN(RD&A)  will  issue  additional  guidance  to  the  various  program  offices  to 
reinforce  this  requirement. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 

A.3.  We  recommend  that  the  AV-8B  Harrier  II  Open  Systems  Core  Avionics 
Requirements  program  manager  develop  and  prepare  a  tailored  Information 
Support  Plan  as  required  by  DoD  Instruction  5000.2  or  obtain  a  properly 
approved  waiver. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  nonconcurred.  He 
stated  that  at  the  time  of  OSCAR’s  development,  there  were  no  specific 
interoperability  requirements  for  the  OSCAR  system  beyond  the  legacy  system  it 
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replaced.  For  these  reasons,  a  command,  control,  communications,  computers, 
and  intelligence  support  plan  waiver  was  granted  on  August  25,  2003.  The  NAE 
was  aware  of  the  waiver  and  agreed  with  it.  The  Navy  believes  that  the  original 
waiver  was  properly  processed  and  approved.  The  rationale  for  the  original 
waiver  request  was  valid  and  remains  valid.  The  OSCAR  program  is  currently 
58-percent  delivered  to  the  Fleet.  Retroactively  completing  an  infonnation 
support  plan  at  this  point  in  the  program’s  life  cycle  would  be  an  inefficient  use  of 
available  resources  with  limited  or  no  benefit  to  the  program. 

Audit  Response.  The  comments  were  not  responsive  to  the  recommendation. 

We  request  that  the  Navy  reconsider  its  position  and  provide  comments  on  the 
final  report.  The  OSCAR  program  should  either  develop  a  tailored  infonnation 
support  plan  or  obtain  a  properly  approved  waiver. 

Although  program  officials  for  OSCAR  obtained  a  command,  control, 
communications,  computers,  and  intelligence  support  plan  waiver  from  the 
Deputy  Assistant  Secretary  of  the  Navy  (Command,  Control,  Communications, 
Computers,  and  Intelligence  and  Space),  the  waiver  did  not  have  the  conect 
approval  authority  and  the  waiver  was  based  on  inappropriate  ORDs.  DoD 
Regulation  5000.2-R,  “Mandatory  Procedures  for  Major  Defense  Acquisition 
Programs  and  Major  Automated  Information  System  Acquisition  Programs,” 

June  10,  2001,  states  that  the  NAE,  with  the  advice  from  the  appropriate  chief 
information  officer,  may  waive  command,  control,  communications,  computers, 
and  intelligence  support  plan  preparation  if  the  requirements  authority  has 
previously  waived  the  requirement  for  an  interoperability  key  performance 
parameter  in  the  ORD.  The  waiver  was  not  approved  by  the  NAE  as  required  by 
DoD  Regulation  5000.2-R.  As  a  result,  the  approval  authority  for  the  waiver  was 
invalid. 

In  addition,  the  approval  of  the  waiver  was  not  based  on  all  of  the  ORDs  being 
used  for  OSCAR.  In  a  Department  of  the  Navy  memorandum,  March  26,  2003, 
the  Office  for  Chief  of  Naval  Operations  supported  waiving  the  requirement  for 
an  OSCAR  command,  control,  communications,  computers,  and  intelligence 
support  plan  for  the  following  reasons. 

•  The  AV-8B  Radar  and  Night  Attack  ORDs,  which  govern  the  OSCAR 
program,  do  not  include  an  interoperability  key  perfonnance  parameter. 

•  The  AV-8B  Radar  and  Night  Attack  ORDs  do  not  include  an  external 
information  exchange  requirements. 

•  Of  the  two  major  components  that  make  up  the  OSCAR  program,  the 
mission  computer  already  has  a  command,  control,  communication, 
computer  and  intelligence  support  plan  waiver,  and  the  weapons  computer 
has  no  connection  to  the  communications  and  information  infrastructure. 

However,  in  our  review  of  the  test  and  evaluation  master  plan,  we  found  that  the 
OSCAR  program  is  testing  requirements  found  in  additional  ORDs  that  are  not 
listed  in  the  memorandum.  Those  additional  requirements  are: 

•  Joint  Direct  Attack  Munition  ORD, 
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•  ARC-210  ORD,  and 


•  AV-8B  Targeting  Pod  Requirements  Letter. 

Therefore,  because  the  waiver  was  not  properly  approved  by  the  NAE  and  the 
memorandum  asking  for  approval  does  not  contain  all  the  requirements  for  the 
OSCAR  system,  we  feel  the  waiver  is  not  valid.  In  addition  to  this  audit,  DoD  IG 
Report  No.  D-2004-109,  “Implementation  of  the  DoD  Management  Control 
Program  for  Navy  Acquisition  Category  II  and  III  Programs,”  August  17,  2004, 
found  that  the  program  sponsor  did  not  develop  an  ORD  for  the  OSCAR  program 
as  required  by  DoD  Instruction  5000.2. 

A.4.  We  recommend  that  the  Surface  Electronic  Warfare  Improvement 
Program  and  the  AN/AES-1  Airborne  Laser  Mine  Detection  System 
program  managers  submit  their  systems  engineering  plans  for  Navy 
Acquisition  Executive  approval  as  required  by  Under  Secretary  of  Defense 
(Acquisition,  Technology,  and  Logistics)  memorandum,  “Policy  for  Systems 
Engineering  in  DoD,”  February  20,  2004. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  SEWIP  systems  engineering  plan  was  approved  on  October  22, 
2006.  In  addition,  and  in  order  to  maintain  the  relevancy  and  effectiveness  of  the 
SEWIP  systems  engineering  plan,  the  ASN(RD&A)  Chief  Engineer  has  tasked 
the  SEWIP  program  manager  to  update  the  systems  engineering  plan  to  reflect 
program  changes  that  occurred  during  its  approval  process. 

The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  also 
stated  that  the  Airborne  Laser  Mine  Detection  System  program  manager  has 
concurred  and  agreed  to  develop  a  systems  engineering  plan  by  March  2007. 

This  systems  engineering  plan  will  be  an  annex  to  a  Global  Organic  Airborne 
Mine  Counter  Measures  systems  engineering  plan.  This  approach  has  been 
planned  with  and  approved  by  the  ASN(RD&A)  Chief  Engineer. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 

A.5.  We  recommend  that  the  Navy  Advanced  Extremely  High  Frequency 
Multiband  Terminal  program  manager  submit  an  updated  acquisition 
program  baseline  to  the  Navy  Acquisition  Executive  within  90  days  of  the 
issuance  of  this  report. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  stated  that  the 
requirement  for  a  new  Navy  Advanced  Extremely  High  Frequency  Multiband 
Terminal  acquisition  program  baseline  was  due  to  the  increase  in  the  scope  and 
cost  of  the  program  to  a  new,  higher  ACAT  level  (from  an  ACAT  II  to  an  ACAT 
IC).  The  program  office  reported  an  acquisition  program  baseline  deviation  and 
continued  to  resubmit  program  deviation  reports  to  update  the  status. 
Development  and  approval  time  of  the  acquisition  program  baseline  was 
unusually  extensive.  The  new  corrected  revision  of  the  acquisition  program 
baseline  has  been  forwarded  to  the  ASN(RD&A)  for  approval. 
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In  addition,  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and 
Budget)  noted  that  the  following  statement  from  the  draft  report  was  incorrect: 
“However,  after  the  program  was  designated  as  an  ACAT  IC,  the  approval  of  the 
acquisition  program  baseline  was  withdrawn.”  He  proposed  this  statement  be 
changed  to:  “After  designation  of  the  program  as  an  ACAT  IC,  the  acquisition 
program  baseline  revision  was  approved.  However,  after  the  approval  it  was 
noted  that  inadvertently  there  were  some  changes  to  the  perfonnance  section  in 
error  that  required  the  acquisition  program  baseline  to  be  corrected  and  resigned. 
The  new  corrected  revision  of  the  APB  has  been  forwarded  to  ASN(RD&A)  for 
approval.” 

Audit  Response.  The  comments  were  responsive  to  the  recommendation.  We 
removed  the  statement  in  question. 
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B.  Operational  Testing  in  the  Milestone 
Decision  Process 

The  NAE  approved  8  of  the  13  programs  we  reviewed  for  either  low-rate 
initial  production  (LRIP)  or  full-rate  production.  The  NAE  approved  five 
of  the  eight  programs  for  LRIP  or  full-rate  production  even  though 
operational  assessment  problems  were  evident.  Three  programs  were 
approved  for  production  without  an  operational  assessment  being 
conducted.  In  addition,  one  program  was  approved  for  LRIP  even  though 
operational  assessment  determined  the  system  was  immature  and  critical 
operational  issues  could  not  be  evaluated,  and  officials  from  one  other 
program  purchased  all  systems  prior  to  completing  operational  testing. 
These  conditions  occurred  for  the  following  reasons. 

•  Program  officials  believed  an  operational  assessment  was  not 
required  by  the  acquisition  decision  memorandum  and  the 
system’s  readiness  to  deploy  could  be  demonstrated  during 
developmental  testing. 

•  The  NAE  delegated  the  LRIP  decision  authority  to  the  PEO 
who  eliminated  the  operational  assessment  due  to  schedule 
delays  and  the  low  risk  from  eliminating  the  operational 
assessment. 

•  Program  officials  did  not  comply  with  the  acquisition  decision 
memorandum  to  complete  an  operational  assessment  prior  to 
the  LRIP  decision. 

•  Program  officials  believed  the  procurement  was  necessary  to 
pennit  an  orderly  transition  to  full-rate  production. 

•  The  NAE  approved  the  purchase  of  all  systems  prior  to  the 
completion  of  operational  test  and  evaluation  to  realize 
potential  cost  savings  and  to  stabilize  the  production  line. 

As  a  result,  the  NAE  did  not  determine  whether  programs  were 
operationally  effective  and  suitable  prior  to  approving  production 
quantities  costing  about  $25  billion.  Without  that  information,  the  Navy 
could  incur  additional  development  work  and  operational  testing  that 
would  lead  to  cost  overruns  and  schedule  delays. 


DoD  and  Navy  Policy 

DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition  System,”  May  12, 
2003,  and  SECNAV  Instruction  5000. 2C,  “Implementation  and  Operation  of  the 
Defense  Acquisition  System  and  the  Joint  Capabilities  Integration  and 
Development  System,”  November  19,  2004,  establish  requirements  for 
operational  test  and  evaluation  and  require  LRIP  quantities  greater  than 
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10  percent  to  be  documented  in  the  acquisition  decision  memorandum.  The 
Defense  Acquisition  University  Test  and  Evaluation  Management  Guide  assists 
the  acquisition  community  in  obtaining  a  better  understanding  of  who  the 
decision  makers  are  and  determining  how  and  when  to  plan  test  and  evaluation 
events. 

DoD  Instruction.  DoD  Instruction  5000.2  states  that  operational  test  and 
evaluation  will  determine  the  effectiveness  and  suitability  of  the  system.  The 
milestone  decision  authority  decides  whether  to  commit  DoD  to  production  at 
Milestone  C.  Milestone  C  authorizes  a  program’s  entry  into  the  production  and 
deployment  phase.  For  major  Defense  acquisition  programs  and  other  Director, 
Operational  Test  and  Evaluation  oversight  programs,  production  and  deployment 
includes  LRIP  and  a  full-rate  production  decision  review.  The  purpose  of  the 
production  and  deployment  phase  is  to  achieve  an  operational  capability  that 
satisfies  the  mission  needs.  Entrance  into  this  phase  depends  on  acceptable 
performance  in  development,  test,  and  evaluation;  operational  assessment;  mature 
software  capability;  and  acceptable  operational  supportability. 

DoD  Instruction  5000.2  further  states  that  LRIP  quantities  should  be  minimized. 
The  milestone  decision  authority  determines  the  LRIP  quantities  for  major 
Defense  acquisition  programs  and  major  systems  at  Milestone  B.  The  milestone 
decision  authority  must  approve  any  increase  in  LRIP  quantity  after  the  initial 
determination.  The  acquisition  strategy  must  document  the  rationale  for 
quantities  exceeding  10  percent  of  total  production. 

Navy  Instruction.  SECNAV  Instruction  5000.2C  states  that  LRIP  quantities 
greater  than  10  percent  of  a  program’s  inventory  objective  must  be  justified  in  the 
acquisition  decision  memorandum  and  acquisition  strategy.  SECNAV 
Instruction  5000. 2C  also  states  that  once  a  program  has  expended  greater  than 
90  percent  of  the  total  program  cost,  the  program  manager  should  request  that  the 
NAE  remove  the  program  from  the  acquisition  category  listing. 

Defense  Acquisition  University  Test  and  Evaluation  Management  Guide.  The 

Defense  Acquisition  University  Test  and  Evaluation  Management  Guide  states 
that  an  operational  assessment  is  an  evaluation  of  the  operational  effectiveness 
and  operational  suitability  made  by  an  independent  operational  test  activity,  with 
user  support  on  other  than  production  systems.  An  operational  assessment 
identifies  significant  trends  noted  in  the  development  effort,  risk  areas,  adequacy 
of  requirements,  and  the  ability  of  the  program  to  support  adequate  operational 
testing.  An  operational  assessment  may  be  conducted  at  any  time  using 
technology  demonstrators,  prototypes,  engineering  development  models,  or 
simulations,  but  will  not  substitute  for  initial  operational  test  and  evaluation 
necessary  to  support  full-rate  production  decisions. 


Operational  Test  and  Evaluation  Overview 


Developmental  Test  and  Evaluation.  SECNAV  Instruction  5000.2C  states  that 
developmental  test  and  evaluation  is  conducted  to  support  risk  management, 
provide  data  on  the  progress  of  system  development,  and  determine  readiness  for 
operational  test  and  evaluation. 


24 


Operational  Test  and  Evaluation.  According  to  the  Defense  Acquisition 
University  Test  and  Evaluation  Management  Guide,  operational  test  and 
evaluation  includes  operational  assessment,  initial  operational  test  and  evaluation 
(referred  to  as  operational  evaluation  by  the  Navy),  and  follow-on  operational  test 
and  evaluation.  DoD  Instruction  5000.2  states  that  for  major  Defense  acquisition 
programs  and  other  Director,  Operational  Test  and  Evaluation  programs, 
production  and  deployment  has  two  major  efforts:  LRIP  and  full-rate  production. 
Entrance  into  this  phase  depends  upon  acceptable  performance  in  development, 
test  and  evaluation,  and  operational  assessment.  In  addition,  the  Instruction 
states,  “Operational  Test  Agency  shall  conduct  an  independent,  dedicated  phase 
of  initial  operational  test  and  evaluation  before  full-rate  production  to  evaluate 
operational  effectiveness  and  suitability.” 

Operational  Assessment.  The  Defense  Acquisition  University  Test  and 
Evaluation  Management  Guide  states  that  operational  assessments  are  part  of 
operational  test  and  evaluation  and  are  conducted  to  detennine  risk  areas, 
adequacy  of  requirements,  and  the  ability  of  the  program  to  support  operational 
testing.  An  operational  assessment  is  normally  conducted  prior  to  Milestone  C  to 
provide  an  early  assessment  of  the  potential  operational  effectiveness  and 
suitability  for  decision  makers  at  decision  points.  These  assessments  attempt  to 
project  the  system’s  potential  to  meet  user  requirements. 


Approval  of  Production  Without  Operational  Assessment 


The  NAE  approved  production  decisions  for  3  of  the  13  programs  reviewed 
without  having  an  operational  assessment  as  required  by  the  DoD  Instruction 
5000.2.  Therefore,  the  NAE  did  not  have  documentation  to  determine  whether 
the  systems  were  operationally  effective  and  suitable  to  meet  user  requirements. 
DoD  Instruction  5000.2  states  that  the  purpose  of  the  production  and  deployment 
phase  is  to  achieve  an  operational  capability  that  satisfies  the  mission  needs. 
Entrance  into  this  phase  depends,  in  part,  on  acceptable  perfonnance  in 
development,  test,  and  evaluation;  operational  assessment;  mature  software 
capability;  no  significant  manufacturing  risks;  acceptable  interoperability;  and 
acceptable  operational  supportability. 

AN/AES-1  Airborne  Laser  Mine  Detection  System.  Program  officials  for  the 
AN/AES-1  Airborne  Laser  Mine  Detection  System  did  not  conduct  an  operational 
assessment  prior  to  the  NAE  approval  of  the  LRIP  acquisition  decision 
memorandum  on  June  14,  2005.  Program  officials  eliminated  the  program’s 
operational  assessment  because  the  acquisition  decision  memorandum  did  not 
require  it  and  program  officials  could  demonstrate  the  system’s  readiness  to 
deploy  during  developmental  testing.  A  developmental  test  report  was  issued  in 
April  2005.  It  stated  that  the  target  depth  and  false  contact  density  perfonnance 
parameters  were  waived  for  Milestone  C.  In  addition,  the  target  depth  key 
performance  parameter  threshold  was  reduced  to  60  percent  of  the  threshold  depth 
for  the  LRIP  decision.  In  an  acquisition  decision  memorandum  dated  June  14, 
2005,  the  NAE  approved  Milestone  C  LRIP  although  the  system  had  shortfalls  in 
three  performance  parameters,  based  on  acceptance  of  the  shortfalls  by  the  Fleet 
and  Resource  Sponsor.  Further,  the  NAE  approved  the  program  for  LRIP 
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quantities  greater  than  the  10-percent  threshold  without  an  explanation  in  the 
acquisition  decision  memorandum  as  required  by  SECNAV  Instruction  5000. 2C. 

Open  Systems  Core  Avionics  Requirements.  Program  officials  did  not 
conduct  an  operational  assessment  prior  to  the  Program  Executive  Officer  Air 
Antisubmarine  Warfare  (ASW),  Assault,  and  Special  Mission  Programs  approval 
of  LRIP  I  on  April  12,  2002.  The  NAE  delegated  the  LRIP  I  decision  authority 
to  the  PEO  Air  ASW,  Assault  and  Special  Mission  Programs  on  March  20,  2000. 
Program  officials  eliminated  the  operational  assessment  from  the  OSCAR  test 
schedule  to  avoid  test  schedule  delays  that  would  occur  because  the  AV-8B 
aircraft  was  not  available  for  testing.  The  program  manager  provided  the  PEO 
with  a  program  deviation  report  dated  May  2,  2001.  It  stated  that  a 
developmental/operational  test  period  had  been  included  in  the  OSCAR  schedule 
so  that  an  early  look  at  the  risky  portions  of  the  software  functionality  could  be 
conducted.  However,  program  officials  felt  that  the  software  functionality  risk 
was  low  and  the  developmental/operational  test  period  was  no  longer  necessary. 
This  software  functionality  would  be  tested  during  operational  evaluation  testing. 
In  addition,  the  program  manager  provided  the  PEO  Air  ASW,  Assault  and 
Special  Mission  Programs  with  a  program  deviation  report  dated  April  4,  2003. 

It  stated  that  the  OSCAR  program  deviated  from  its  approved  baseline  dated 
February  7,  2002,  to  correct  anomalies  discovered  during  OSCAR  operational 
evaluation  testing.  This  resulted  in  a  1-year  delay  in  the  completion  of 
operational  evaluation  testing  from  July  2003  to  July  2004.  Had  program 
officials  conducted  an  operational  assessment,  these  anomalies  could  have  been 
detected  prior  to  entering  into  operational  evaluation  testing. 

In  a  report  dated  July  20,  2004,  the  Commander,  Operational  Test  and  Evaluation 
Force  found  the  OSCAR  program  to  be  operationally  effective  but  not 
operationally  suitable.  On  August  16,  2004,  the  NAE  issued  an  acquisition 
decision  memorandum  approving  full-rate  production.  In  the  memorandum,  the 
NAE  stipulated  all  suitability  issues  in  the  operational  test  report  must  be 
corrected  and  that  the  Commander,  Operational  Test  and  Evaluation  Force  must 
verify  the  corrections  prior  to  fleet  introduction  and  initial  operational  capability. 
Program  office  and  NAE  staff  did  not  have  documentation  to  show  that  the 
corrections  were  made  to  OSCAR  and  that  the  corrections  were  verified  by  the 
Commander,  Operational  Test  and  Evaluation  Force.  As  a  result,  we  were  unable 
to  determine  whether  the  corrections  were  made  to  OSCAR  or  that  the  corrections 
were  verified  by  the  Commander  Operational  Test  and  Evaluation  Force.  A 
former  member  of  the  OSCAR  Overarching  Integrated  Product  Team  advised  us 
that  OSCAR  was  released  to  the  fleet  without  any  evidence  the  corrections  were 
made. 

MH-60R  Multi-Mission  Helicopter.  Program  officials  for  the  MH-60R  did  not 
complete  an  operational  assessment  prior  to  LRIP  II  approval.  The  program 
changed  its  acquisition  strategy  from  a  remanufacture  approach  to  a  new-built 
approach  starting  with  LRIP  II,  in  an  acquisition  decision  memorandum  dated 
March  14,  2002.  In  addition,  the  acquisition  decision  memorandum  required 
completion  of  an  operational  assessment  with  a  conclusion  that  the  MH-60R  was 
potentially  operationally  effective  and  suitable  prior  to  the  approval  of  LRIP  II. 
The  Commander,  Operational  Test  and  Evaluation  Force  conducted  an 
operational  assessment  of  the  MH-60R  from  June  2,  2003,  through 
September  3,  2003.  However,  prior  to  completion  of  the  operational  assessment, 
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the  program  office  terminated  the  testing  due  to  poor  perfonnance  of  the  radar, 
electronic  support  measures,  and  acoustic  subsystems.  The  NAE  approved  the 
program  for  LRIP  II  on  December  15,  2003,  even  though  an  operational 
assessment  was  not  completed,  as  required  by  DoD  Instruction  5000.2  and  the 
acquisition  decision  memorandum  March  14,  2002.  In  addition,  no  justification 
was  included  in  the  acquisition  decision  memorandum  for  proceeding  with  LRIP 
without  the  required  operational  assessment. 

Further,  a  second  operational  assessment  was  conducted  from  October  through 
December  2004.  The  operational  assessment  report,  dated  March  18,  2005,  stated 
that  the  system  did  not  meet  all  threshold  values  for  operational  effectiveness  and 
operational  suitability.  Although  the  report  included  a  listing  of  recommendations 
that,  if  not  addressed,  could  result  in  the  failure  of  operational  evaluation  testing, 
the  Commander,  Operational  Test  and  Evaluation  Force  recommended  continued 
program  development.  In  addition,  the  NAE  approved  LRIP  III,  in  an  acquisition 
decision  memorandum  April  5,  2005. 


Approval  of  Low-Rate  Initial  Production  With  Known 
Deficiencies 


AN/APG-79  Active  Electronically  Scanned  Array  Radar  System.  Despite 
risks  and  deficiencies  identified  in  operational  assessment,  the  NAE  approved 
acquisition  decision  memorandums  for  the  procurement  of  84  AN/APG-79  Active 
Electronically  Scanned  Array  Radar  Systems  from  July  2003  through 
February  2006.  Additionally,  these  procurements  represented  20  percent  of  the 
total  number  of  systems  to  be  purchased.  However,  the  February  2006 
acquisition  decision  memorandums  did  not  contain  justification  to  exceed  the  10- 
percent  LRIP  guideline  as  required  by  SECNAV  Instruction  5000. 2C.  The 
Commander,  Operational  Test  and  Evaluation  Force  issued  an  Operational 
Assessment  Test  Report  for  the  AN/APG-79  Active  Electronically  Scanned  Array 
Radar  System  on  April  4,  2006.  The  report  stated  that  the  system  did  not  meet  or 
exceed  threshold  values.  The  Commander,  Operational  Test  and  Evaluation 
Force  recommended  the  continued  development  of  the  system  and  an  additional 
developmental  test-assist  period  once  deficiencies  noted  in  the  report  have  been 
corrected.  However,  the  NAE  approved  LRIP  even  though  operational 
assessments  disclosed  that  the  system  was  not  mature,  all  critical  operational 
issues  could  not  be  evaluated,  and  the  system  did  not  meet  or  exceed  threshold 
values. 


Procuring  All  Systems  Prior  to  Operational  Evaluation 


AN/SPY-1D(V)  Radar  Upgrade.  The  NAE  approved  procurement  of  all 
22  AN/SPY- 1D(V)  systems  from  January  1997  through  December  2003,  while 
the  program  was  in  LRIP.  The  NAE  stated  that  because  the  AN/SPY- 1D(V) 
system  was  an  integral  part  of  the  DDG-5 1  ship  structure,  it  was  essential  that  its 
acquisition  strategy  support  the  DDG-5 1  multiyear  ship  construction  schedule. 
The  NAE  decided  that  the  advantages  of  the  LRIP  approach,  which  included  the 
opportunity  to  realize  cost  savings  and  stabilize  essential  elements  of  the 


27 


production  base,  outweighed  the  low  risks  associated  with  the  decision  to 
proceed. 

DoD  Instruction  5000.2  and  SECNAV  Instruction  5000. 2C  require  that  an 
independent  operational  test  and  evaluation  (referred  to  by  the  Navy  as 
operational  evaluation)  be  conducted  prior  to  the  full-rate  production  decision. 
The  Commander,  Operational  Test  and  Evaluation  Force  conducted  an 
operational  evaluation  of  the  AN/SPY- 1D(V)  system  in  March,  October,  and 
November  2005  and  issued  an  Operational  Evaluation  Test  Report  on  April  5, 
2006.  The  test  was  conducted  and  the  report  was  issued  after  approval  was  given 
to  procure  all  22  AN/SPY-1D(V)  systems.  The  test  report  stated  that  the  system 
did  not  meet  all  suitability  threshold  values.  However,  the  Commander, 
Operational  Test  and  Evaluation  Force  recommended  continued  fleet  introduction 
of  the  AN/SPY- 1D(V)  system  with  priority  given  to  the  correction  and 
verification  of  the  suitability  issues  noted  in  the  report.  According  to  the  program 
manager,  correction  and  verification  of  the  suitability  issues  discussed  in  the 
Operational  Evaluation  Test  Report  is  ongoing.  The  AN/SPY- 1D(V)  system  was 
scheduled  for  a  Milestone  C  full-rate  production  decision  review  between 
July  2006  and  January  2007.  The  program  office  staff  anticipates  requesting  that 
the  NAE  remove  the  program  from  the  acquisition  category  listing  and  close  out 
the  program  without  a  Milestone  C  full-rate  production  decision  meeting.  At  the 
time  of  our  review,  program  office  staff  stated  they  are  working  on  the  closeout. 

In  approving  the  procurement  of  all  22  AN/SPY- 1D(V)  systems  in  LRIP,  the 
NAE  made  a  full-rate  production  decision  without  conducting  an  operational 
evaluation  of  the  AN/SPY- 1D(V)  system  to  ensure  the  system  was  operationally 
effective  and  suitable  and  met  the  users  needs. 


Related  Audit  Coverage 


The  DoD  Inspector  General  (IG)  issued  Report  No.  D-2006-001,  “Audit  of  the 
Common  Submarine  Radio  Room,”  October  3,  2005,  which  stated  that  the 
program  manager  did  not  schedule  an  operational  assessment  of  the  Common 
Submarine  Radio  Room  prior  to  the  June  2005  LRIP  decision  program  review. 

The  DoD  IG  recommended  that  the  program  manager  delay  the  review  of  LRIP 
until  he  obtains  an  operational  assessment  that  states  the  Common  Submarine 
Radio  Room  is  operationally  effective  and  suitable.  The  Navy  nonconcurred  with 
the  recommendation;  however,  during  mediation  the  Commander,  Operational 
Test  and  Evaluation  Force  agreed  to  prepare  an  operational  assessment  report 
which  satisfies  the  requirements  of  DoD  Instruction  5000.2.  In  addition,  the  Navy 
agreed  to  provide  the  final  operational  assessment  report  to  the  milestone  decision 
authority  prior  to  the  FY  2006  program  review  of  the  Common  Submarine  Radio 
Room. 


Conclusion 


Operational  assessments  are  conducted  early  in  the  program  to  provide  insight 
into  potential  operational  problems  and  progress  toward  meeting  desired 
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operational  effectiveness  and  suitability  capabilities.  Programs  should  not  be 
approved  to  enter  the  production  and  deployment  phase  of  the  acquisition  process 
until  the  system  has  demonstrated  during  operational  testing  that  the  system  is 
potentially  operationally  effective  and  suitable  and  will  meet  the  users’  needs. 
Approving  systems  to  enter  LRIP  and  full-rate  production  before  the  system  has 
demonstrated  that  it  is  potentially  operationally  effective  and  suitable  could  result 
in  systems  being  produced  that  will  not  meet  the  users’  needs  and  require 
additional  funds  to  retrofit  the  systems  that  have  already  been  produced.  In 
addition,  milestone  decisions  related  to  deviations  from  LRIP  or  full-rate 
production  buys,  operational  assessment,  and  initial  operational  test  and 
evaluation  requirements  should  be  documented  in  the  acquisition  decision 
memorandum  to  document  approval  of  the  actions  taken. 


Management  Comments  on  the  Finding  and  Audit  Response 


Summaries  of  management  comments  on  the  finding  and  our  audit  responses  are 
in  Appendix  H. 


Recommendations,  Management  Comments,  and  Audit 
Response 


Revised,  Added,  Deleted,  and  Renumbered  Recommendations,  The  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  provided  management 
comments  on  behalf  of  the  ASN(RD&A).  We  added  Recommendation  B.2.  and 
deleted  B.6.  We  renumbered  B.2.,  B.3.,  and  B.4.  as  B.3.,  B.4.,  and  B.5. 
accordingly.  We  revised  Recommendation  B.3.  to  clarify  the  intent  of  the 
recommendation  and  Recommendation  B.4.  to  correspond  to  DoD  Instruction 
5000.2  requirements  regarding  LRIP. 

B.  We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition): 

1.  Require  program  offices  to  schedule  and  complete  an  operational 
assessment  prior  to  the  low-rate  initial  production  milestone  decision  in 
accordance  with  DoD  Instruction  5000.2. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  nonconcured.  The 
Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  believes 
SECNAV  Instruction  5000.2C  guidance  regarding  operational  assessment 
activities  prior  to  the  LRIP  is  consistent  with  the  DoD  Instruction  5000.2.  He 
discussed  four  references  within  the  SECNAV  Instruction  5000.2C  regarding 
waivers  and  deferrals.  Additionally,  the  Department  of  the  Navy  Acquisition  and 
Capabilities  Guidebook  states  that  the  milestone  decision  authority  should 
determine  whether  operational  test  and  evaluation  is  required  prior  to  LRIP  for 
non-Office  of  the  Secretary  of  Defense  Test  and  Evaluation  oversight  programs. 
The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  believes 
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that  this  guidance  gives  the  milestone  decision  authority  the  discretion  to 
determine  whether  developmental  or  integrated  test  data  and  reports  are  adequate 
to  make  an  LRIP  decision  or  if  additional  operational  assessment  is  needed.  In 
addition,  he  stated  that  forcing  such  programs  to  include  an  operational 
assessment  in  the  test  and  evaluation  strategy,  when  program  risk  can  be 
adequately  managed  without  the  use  of  an  operational  assessment,  is  inefficient 
and  negates  the  Chief  of  Naval  Operation’s  challenge  to  the  Navy  to  streamline 
test  and  evaluation.  The  Navy  guidance  allows  the  necessary  flexibility  to 
achieve  cost,  schedule,  and  performance.  The  Deputy  Assistant  Secretary  of  the 
Navy  (Management  and  Budget)  believes  that  the  current  guidance  is  adequate 
and  should  not  be  changed. 

Audit  Response.  The  comments  were  not  responsive  to  the  recommendation. 

We  request  that  the  Navy  reconsider  its  position  and  provide  comments  in 
response  to  the  final  report. 

Allowing  the  milestone  decision  authority  to  determine  whether  operational  test 
and  evaluation  is  required  prior  to  LRIP  for  certain  programs  is  not  in  compliance 
with  requirements  in  DoD  Directive  5000.1  and  DoD  Instruction  5000.2.  DoD 
Directive  5000.1  requires  that  test  and  evaluation  be  structured  to  provide 
essential  information  to  decision  makers  to  determine  whether  systems  are 
operationally  effective,  suitable,  survivable,  and  safe  for  the  intended  use.  It  also 
states  that  the  policies  in  this  Directive  apply  to  all  acquisition  programs.  DoD 
Instruction  5000.2  implements  policies  in  DoD  Directive  5000. 1 .  It  establishes 
the  requirement  that  before  systems  enter  into  LRIP,  the  system  must  demonstrate 
acceptable  performance  in  development,  test,  and  evaluation  and  undergo  an 
operational  assessment. 

The  four  references  from  the  SECNAV  5000. 2C  discussed  in  the  management 
comments  do  not  address  the  requirement  for  program  offices  to  schedule  and 
complete  an  operational  assessment  prior  to  LRIP.  Rather,  the  references  discuss 
waiver  and  deferral  requirements.  In  fact,  the  reference  to  deferrals  states  that 
deferrals  will  not  normally  be  granted  for  an  operational  assessment  prior  to 
initial  operational  test  and  evaluation.  The  reference  to  waivers  states  that 
waivers  do  not  change  or  delay  any  testing  or  evaluation  of  a  system. 

One  of  the  programs  that  did  not  complete  an  operational  assessment  before  the 
LRIP  decision,  the  MH-60R  Multi-Mission  Helicopter,  was  an  Office  of  the 
Secretary  of  Defense  Test  and  Evaluation  oversight  program.  In  the  March  14, 
2002,  acquisition  decision  memorandum,  the  milestone  decision  authority 
required  completion  of  an  operational  assessment  with  a  conclusion  that  the 
MH-60R  was  potentially  operationally  effective  and  suitable  before  the  decision 
review  to  approve  LRIP  II.  Regardless,  the  milestone  decision  authority 
approved  LRIP  II  even  though  the  program  office  had  not  conducted  an 
operational  assessment. 

Further,  two  other  programs  which  did  not  undergo  an  operational  assessment 
before  the  LRIP  decision  subsequently  experienced  program  delays  and  shortfalls 
in  demonstrating  the  ability  to  satisfy  key  performance  parameters. 

•  On  the  OSCAR  program,  program  officials  canceled  the  requirement 
to  include  a  developmental  and  operational  test  period  to  look  at  the 
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risk  associated  with  the  software  functionality  which  would  help 
resolve  the  breach  from  the  approved  baseline.  The  program  officials 
felt  that  software  risk  was  low  and  could  be  tested  during  operational 
evaluation  testing.  Subsequent  to  the  LRIP  decision,  the  program 
again  deviated  from  its  approved  baseline  to  correct  anomalies 
discovered  during  operational  evaluation  testing,  which  resulted  in  a 
1-year  delay  for  the  completion  of  operational  evaluation  testing. 

•  Program  officials  for  the  AN/AES-1  Airborne  Laser  Mine  Detection 
System  eliminated  the  operational  assessment  because  the  acquisition 
decision  memorandum  did  not  require  an  operational  assessment. 
Program  officials  believed  they  could  demonstrate  the  system’s 
readiness  to  deploy  during  developmental  testing.  Subsequent  to  the 
LRIP  decision,  the  ASN(RD&A)  had  to  waive  two  key  performance 
parameters.  Another  key  performance  parameter  was  reduced  to 
60  percent  of  the  threshold  because  the  Fleet  and  Resource  Sponsor 
accepted  the  current  performance  limitations  and  the  proposed  strategy 
to  bring  the  program  to  full  performance  for  developmental  testing. 
This  decision  resulted  in  an  approved  LRIP  with  shortfalls  in  these 
three  key  perfonnance  parameters. 

Conducting  operational  assessments  as  required  before  the  LRIP  decision  would 
have  provided  the  milestone  decision  authority  with  better  infonnation 
concerning  program  risks.  Program  officials  could  have  avoided  costs  associated 
with  retrofitting  LRIP  units  already  produced. 

Because  the  Navy  Acquisition  Guidebook  is  not  in  compliance  with  requirements 
in  DoD  Directive  5000.1  and  DoD  Instruction  5000.2,  we  added 
Recommendation  B.2.  to  update  the  Navy  Acquisition  Guidebook. 

2.  Update  the  Navy  Acquisition  Guidebook  to  conform  to  the 
requirements  in  DoD  Instruction  5000.2.  Specifically,  the  Navy  Acquisition 
Guidebook  should  require  that  before  acquisition  programs  enter  low-rate 
initial  production,  the  system  must  demonstrate  acceptable  performance  in 
development  test  and  evaluation  and  undergo  an  operational  assessment. 

3.  Document  in  the  acquisition  decision  memorandum  quantities 
exceeding  the  10-percent  low-rate  initial  production  guideline. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred  with 
comment.  He  stated  that  the  ASN(RD&A)  will  document  in  the  acquisition 
decision  memorandum  justification  for  exceeding  the  10-percent  LRIP  guideline. 
The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that 
this  justification  has  been  reflected  in  recent  acquisition  decision  memorandums 
and  the  statutorily  required  Selected  Acquisition  Reports.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  also  stated  that  the  LRIP  statute 
(section  2400(a),  title  10,  United  States  Code)  does  not  set  10  percent  as  an  LRIP 
limitation.  Rather,  10  percent  is  a  guideline  and  a  threshold  for  which  further 
explanation  and  documentation  in  the  Selected  Acquisition  Report  is  required  for 
AC  AT  I  programs. 


31 


Audit  Response.  In  response  to  management  comments,  we  revised 
Recommendation  B.3.  to  indicate  that  10  percent  is  an  LRIP  guideline  rather  than 
a  limitation.  The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and 
Budget)  comments  are  responsive. 

4.  Assess  the  cost  and  benefits  of  a  break  in  production  versus 
continuing  annual  buys  when  the  program  office  expects  to  exceed  the  low- 
rate  initial  production  quantity  approved  at  Milestone  B. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  nonconcurred.  He 
stated  that  the  LRIP  statute  (section  2400(a),  title  10,  United  States  Code)  does 
not  set  10  percent  as  an  LRIP  limitation.  Rather,  10  percent  is  a  guideline  and  a 
threshold  for  which  further  explanation  and  documentation  in  the  Selected 
Acquisition  Report  is  required  for  ACAT  I  programs.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  also  stated  that  the  LRIP  statute 
requires  the  milestone  decision  authority  to  establish  the  LRIP  quantity  at 
Milestone  B  and  the  Secretary  of  Defense  to  report  on  the  LRIP  quantity  in  the 
first  system  assessment  for  ACAT  I  programs.  The  Deputy  Assistant  Secretary  of 
the  Navy  (Management  and  Budget)  further  stated  that  both  of  these  events 
typically  occur  before  the  system  is  ready  to  proceed  to  LRIP. 

Additionally,  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and 
Budget)  stated  that  the  Commander,  Operational  Test  and  Evaluation  Force’s 
evaluation  of  a  system  is  restricted  to  whether  or  not  the  system  under  test 
satisfies  the  requirements  of  the  capabilities  documents.  Test  and  evaluation  is 
only  one  consideration  by  the  milestone  decision  authority  in  support  of  a 
production  decision.  The  Deputy  Assistant  Secretary  of  the  Navy  (Management 
and  Budget)  further  stated  that  the  Commander,  Operational  Test  and  Evaluation 
Force  has  neither  connection  to,  nor  understanding  of,  all  acquisition 
considerations.  Therefore,  the  Commander  is  unable  to  render  a  complete  and 
informed  recommendation  to  the  milestone  decision  authority  concerning  LRIP 
and  should  limit  its  reporting  to  system  perfonnance,  leaving  acquisition  officials 
to  resolve  production  decisions. 

Audit  Response.  We  revised  Recommendation  B.4.  to  correspond  to  the 
requirement  of  DoD  Instruction  5000.2  regarding  LRIP. 

DoD  Instruction  5000.2  states  that  LRIP  is  to  result  in  completion  of 
manufacturing  development  to  ensure  adequate  and  efficient  manufacturing 
capability.  LRIP  also  produces  the  minimum  quantity  necessary  to  provide 
production  articles  for  initial  operational  test  and  evaluation.  In  addition,  DoD 
Instruction  5000.2  states,  “when  approved  LRIP  quantities  are  expected  to  be 
exceeded  because  the  program  has  not  yet  demonstrated  readiness  to  proceed  to 
full-rate  production,  the  milestone  decision  authority  shall  assess  the  cost  and 
benefits  of  a  break  in  production  versus  continuing  annual  buys.”  We  request  that 
the  Navy  provide  comments  to  the  revised  recommendation  in  response  to  the 
final  report. 

5.  Instruct  the  program  manager  for  AV-8B  Harrier  II  Open  Systems 
Core  Avionics  Requirements  to  provide  documentation  that  the  suitability 
issues  in  the  Operational  Test  Report  for  the  AV-8B  Harrier  II  Open 
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Systems  Core  Avionics  Requirements  were  corrected  and  that  the 
corrections  were  verified  by  the  Commander,  Operational  Test  and 
Evaluation  Force. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  deficiencies  cited  in  the  operational  test  report  for  the  AV-8B 
Harrier  II  Open  Systems  Core  Avionics  Requirements  were  reevaluated  during 
subsequent  operational  testing  of  another  program.  The  Commander,  Operational 
Test  and  Evaluation  Force  stated  that  the  four  major  deficiencies  in  AV-8B 
Harrier  II  Open  Systems  Core  Avionics  Requirements  were  corrected. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation.  We 
reviewed  a  copy  of  the  report,  Commander,  Operational  Test  and  Evaluation 
Force  3980  (S0195-08-OT-IIIA-SCS-H2.0)  Ser  573/S001,  January  9,  2006.  We 
determined  that  the  deficiencies  cited  in  the  operational  test  report  were  corrected. 

6.  Require  the  program  manager  for  the  AN/APG-79  Active 
Electronically  Scanned  Array  Radar  System  to  obtain  a  determination  from 
the  Commander,  Operational  Test  and  Evaluation  Force  that  the 
AN/APG-79  Active  Electronically  Scanned  Array  Radar  System  is 
potentially  operationally  effective  and  suitable  before  approving  any  further 
production  buys. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  nonconcurred.  He 
stated  that  the  AN/APG-79  Active  Electronically  Scanned  Array  Radar  System 
operational  evaluation  test  was  completed  in  November  2006,  and  the  report  will 
be  published  in  February  2007.  He  further  stated  that  the  beyond  LRIP  report 
will  be  issued  in  March  2007,  with  the  full-rate  production  decision  planned  for 
April  2007.  Additionally,  the  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  stated  that  the  acquisition  decision  memorandum  for  a 
fourth  LRIP  provides  exit  criteria  requiring  the  system  to  be  operationally 
effective  and  suitable. 

Audit  Response.  Although  the  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  nonconcurred,  the  comments  were  responsive  to  the 
recommendation.  We  agree  that  completion  of  the  AN/APG-79  Active 
Electronically  Scanned  Array  Radar  operational  evaluation  testing  and  the 
establishment  of  exit  criteria  for  requiring  an  operationally  effective  and  suitable 
system  prior  to  the  full-rate  production  decision  meets  the  intent  of  our 
recommendation. 

7.  Instruct  the  program  manager  for  the  AN/SPY-1D(V)  Radar 
Upgrade  System  to  provide  documentation  that  the  suitability  issues  in  the 
Operational  Test  Report  for  the  AN/SPY-1D(V)  Radar  Upgrade  System  were 
corrected  and  that  the  corrections  were  verified  by  the  Commander, 
Operational  Test  and  Evaluation  Force.  In  addition,  we  recommend  that 
closeout  of  the  program  not  be  approved  until  the  requested  documentation 
is  provided. 
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Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  program  office  is  implementing  a  plan  to  resolve  outstanding 
suitability  deficiencies  identified  in  the  operational  evaluation  report  prior  to  the 
Milestone  III  full-rate  production  decision  review. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 
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C.  Reporting  in  Dashboard 

Program  managers  for  2  of  the  13  Navy  programs  reviewed  did  not  update 
program  information  in  the  Dashboard  database  on  a  quarterly  basis. 
Additionally,  program  managers  for  evolutionary  acquisition  programs 
could  not  report  each  increment  separately  in  Dashboard.  These 
conditions  existed  because  the  NAE  had  not  issued  policy  mandating  the 
use  of  Dashboard,  and  Dashboard  did  not  provide  program  managers  with 
the  capability  to  separately  report  program  information  for  each  increment 
of  evolutionary  acquisition  programs.  To  be  an  effective  program 
management  tool,  Dashboard  should  allow  Navy  officials  to  interpret 
current  program  status  and  distinguish  program  information  for  each 
increment  of  evolutionary  acquisition  programs. 


Navy  Program  Management  Database 


Dashboard.  The  NAE  has  developed  a  Web-based  database  called  Dashboard. 
Dashboard  is  a  repository  of  data  that  documents  and  supports  the  completion  of 
all  acquisition  program  statutory  and  regulatory  reporting  requirements.  The 
purpose  of  Dashboard  is  to  provide  Secretary  of  the  Navy,  ASN(RD&A),  Chief  of 
Naval  Operations,  Systems  Commands,  PEOs,  and  the  program  manager  an 
automated  tool  that  helps  manage  Navy  programs.  Specifically,  Dashboard  is 
intended  to  enable  Navy  officials  to  monitor  program  progress  between  milestone 
decisions.  Dashboard  was  originally  developed  for  Navy  ACAT  I  and  II 
programs  in  December  2003.  The  database  has  evolved  to  include  ACAT  III 
programs  with  the  plan  of  adding  ACAT  IV  program  data.  According  to  Navy 
officials,  USD(AT&L)  has  directed  that  Dashboard  information  feed  into  another 
information  system  managed  by  USD(AT&L),  entitled  Defense  Acquisition 
Management  Information  Retrieval.  In  the  future,  USD(AT&L)  plans  to  replace 
the  Defense  Acquisition  Executive  Summary  reports  with  the  Defense 
Acquisition  Management  Information  Retrieval  system. 


Dashboard  Input  User  Manual 


According  to  the  NAE  Dashboard  Input  User  Manual,  program  managers  are 
required  to  input  original  program  information  into  the  database  every  quarter. 
However,  the  Dashboard  Input  User  Manual  is  not  formally  signed  guidance.  As 
a  result,  it  is  not  mandatory  or  enforceable.  Before  official  submission  to  the 
NAE,  the  program-assigned  PEOs  review  and  approve  the  data.  Each  Navy 
program  reported  in  Dashboard  is  assigned  a  quarterly  reporting  schedule.  The 
quarterly  schedules  for  ACAT  I  programs  are  based  on  the  reporting  requirement 
in  the  Defense  Acquisition  Executive  Summary. 
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Program  information  reported  in  Dashboard  includes: 

•  10  program  assessment  areas;5 

•  milestone  and  acquisition  decision  memorandum  information; 

•  cost,  funding,  and  contracting  information; 

•  key  performance  parameters; 

•  joint  capability  areas; 

•  PEO  and  program  manager  comments;  and 

•  regulatory  and  statutory  document  status. 


Dashboard  Updates 


During  the  audit,  we  were  given  access  to  the  Dashboard  system  and  reviewed 
Dashboard  program  information  for  the  13  Navy  programs.  Program  officials  for 
2  of  the  13  programs  did  not  update  Dashboard  according  to  their  quarterly 
schedules.  However,  quarterly  Dashboard  reporting  applies  to  all  ACAT 
programs  we  reviewed. 

Quarterly  Reporting.  During  our  review  of  the  Dashboard  system,  we 
discovered  that  the  Dashboard  reports  for  the  AN/AES-1  Airborne  Laser  Mine 
Detection  System  and  OSCAR  program  were  not  current.  At  that  time,  program 
officials  had  not  updated  Dashboard  information  for  the  AN/AES-1  Airborne 
Laser  Mine  Detection  System  in  7  months.  Dashboard  was  not  updated  quarterly 
even  though  an  overall  program  status  report  indicated  that  some  event,  action,  or 
delay  was  impairing  progress  on  major  program  objectives.  Dashboard  color 
coding  guidelines  encourage  early  reporting  for  programs  with  this  status. 
Similarly,  during  our  review  of  the  Dashboard  report  for  the  OSCAR  program, 
Dashboard  had  not  been  updated  in  over  6  months. 

Reporting  Reliability.  Regardless  of  ACAT  level,  if  a  program  is  not  reporting 
quarterly,  NAE  staff  contacts  program-assigned  Deputy  Assistant  Secretaries  of 
the  Navy,  PEOs,  and  program  managers  to  encourage  them  to  report  on  time. 
However,  NAE  staff  stated  that  the  Navy  has  no  formal  written  policy  mandating 
the  use  of  Dashboard  with  quarterly  updates  for  all  ACAT  programs.  Without  a 
written  policy,  the  Secretary  of  the  Navy,  NAE,  Chief  of  Naval  Operations, 
Systems  Commands,  PEOs,  and  the  program  managers  cannot  be  assured  that  the 
information  in  Dashboard  accurately  reflects  the  status  of  the  programs.  As  a 
result,  the  Navy  is  not  using  Dashboard  to  its  fullest  potential  as  a  tool  for 
monitoring  program  progress.  Additionally,  because  USD(AT&L)  plans  to 
extract  information  from  Dashboard  to  populate  the  Defense  Acquisition 


5  Performance  characteristics,  test  and  evaluation,  logistic  requirements  and  readiness  objectives,  cost, 
binding,  schedule,  contracts,  production,  management  structure,  and  interoperability. 
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Management  Infonnation  Retrieval  system,  it  is  important  that  the  information  be 
current. 

Defense  Acquisition  Executive  Summary  Reports.  Defense  Acquisition 
Executive  Summary  reports  are  required  for  ACAT  I  programs  and  are  submitted 
to  the  NAE  no  later  than  the  15  th  day  of  the  program’s  designated  quarterly 
reporting  month.  The  Defense  Acquisition  Executive  Summary  is  a  multipart 
document  that  reports  program  infonnation  and  assessments;  NAE,  PEO,  and 
program  manager  comments;  and  funding  data.  The  Defense  Acquisition 
Executive  Summary  provides  an  early-warning  report  to  the  USD(AT&L),  and 
describes  actual  program  problems,  warns  of  potential  program  problems,  and 
describes  mitigating  actions  taken  or  planned.  Because  Dashboard  is  to  provide 
program  data  to  the  Defense  Acquisition  Management  Information  Retrieval 
system,  which  will  replace  the  need  for  Defense  Acquisition  Executive  Summary 
reports  in  the  future,  it  is  critical  that  program  information  be  updated  on  a 
quarterly  basis.  Under  the  current  process,  outdated  information  could  be 
provided  to  the  USD(AT&L),  which  would  inhibit  effective  oversight  for  ACAT  I 
programs.  The  Navy  should  issue  policy  that  mandates  all  program  offices  update 
Dashboard  on  a  quarterly  basis. 


Evolutionary  Acquisition  Programs 


Of  the  13  Navy  programs  reviewed,  2  (SEWIP  and  Common  Link  Integration 
Processing  [CLIP])  were  evolutionary  acquisition  programs.  Although 
Dashboard  allows  the  program  manager  for  programs  to  enter  data  that  tracks 
milestone  information,  Dashboard  does  not  specifically  distinguish  information 
about  documents  for  each  increment.  As  a  result,  program  managers  must 
overwrite  existing  program  data  to  update  Dashboard  when  future  blocks 
approach  milestone  decisions.  This  process  makes  it  difficult  for  Navy  officials 
to  distinguish  data  for  each  increment  of  an  evolutionary  acquisition  program, 
which  affects  how  the  NAE  interprets  Dashboard  reports  and  manages  the 
programs. 

According  to  DoD  Instruction  5000.2,  DoD  prefers  to  procure  weapon  systems 
using  an  evolutionary  acquisition  strategy.  An  evolutionary  approach  delivers  the 
capability  in  increments,  which  allows  the  Navy  to  field  a  system  sooner.  A  Navy 
program  can  be  separately  developed  in  several  increments,  such  as  increments  A, 
B,  and  C,  and  further  broken  down  into  subincrements,  such  as 
subincrements  1  and  2.  The  figure  demonstrates  how  SEWIP  uses  the 
evolutionary  acquisition  approach  to  structure  its  program.  Each  increment  and 
subincrement  will  have  separate  milestone  decisions  and  program  documentation. 
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Increments 


Structure  of  the  SEWIP  Program 

Surface  Electronic  Warfare  Improvement  Program.  The  Dashboard 
report  for  SEWIP  includes  data  for  two  of  its  subincrements:  Electronic 
Surveillance  Enhancement  and  Improved  Control  and  Display.  Both  of  these 
subincrements  have  been  approved  for  Milestone  C  and  the  infonnation  for  both 
subincrements  has  been  reported  to  Dashboard.  However,  the  Dashboard  design 
did  not  allow  the  program  manager  for  SEWIP  to  report  infonnation  on  the  two 
subincrements  in  the  appropriate  areas.  The  Dashboard  report  shows  the 
Electronic  Surveillance  Enhancement  acquisition  strategy  date  under  a  column  for 
Milestone  C  information.  However,  because  Dashboard  does  not  provide  a  place 
to  report  the  Improved  Control  and  Display  Milestone  C  infonnation,  the 
acquisition  strategy  date  is  reported  in  a  section  for  the  full-rate  production 
information.  This  case  illustrates  the  need  for  the  NAE  to  modify  Dashboard  to 
separate  data  for  each  increment. 

Common  Link  Integration  Processing.  The  Navy  is  also  procuring  the 
CLIP  program  using  an  evolutionary  acquisition  approach.  Because  the 
Dashboard  design  does  not  provide  separate  areas  to  enter  information  for  each 
increment,  the  program  manager  for  CLIP  tried  to  clarify  by  labeling  each  entry 
as  increment  1  or  increment  2  data.  However,  when  increment  2  progresses  to 
Milestone  C,  the  program  manager  will  have  to  overwrite  the  increment  1  data  for 
Milestone  C.  In  other  words,  the  program  manager  will  have  to  overwrite 
existing  information  with  new  data  to  update  Dashboard. 


Dashboard  Modification 


To  help  Navy  officials  manage  Navy  programs,  the  NAE  needs  to  reconsider  the 
current  design  of  Dashboard  and  the  implications  on  evolutionary  acquisition 
programs.  As  demonstrated  by  SEWIP  and  CLIP,  the  current  design  will  cause  a 
loss  of  historical  data  for  past  increments.  Additionally,  Navy  officials  will  not 
be  able  to  distinguish  between  important  milestone  decision  dates  and  data  for 
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each  increment.  Combining  data  for  different  increments  prevents  Navy  officials 
from  having  full  knowledge  of  the  program  historical  data.  If  the  NAE  modifies 
Dashboard  to  distinguish  program  data  for  different  increments,  the  Navy  will 
preserve  historical  and  current  program  information  in  Dashboard  and  allow 
evolutionary  acquisition  programs  to  clearly  report  information  for  different 
program  increments. 


Recommendations,  Management  Comments,  and  Audit 
Response 


C.  We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition): 

1.  Issue  official  guidance  to  Deputy  Assistant  Secretaries  of  the  Navy, 
program  executive  officers,  and  program  managers  mandating  the  use  of 
Dashboard  with  quarterly  updates  for  all  ACAT  programs. 

Management  Comments,  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  ASN(RD&A)  will  issue  official  guidance  to  the  Deputy  Assistant 
Secretaries  of  the  Navy,  PEOs,  and  program  managers  mandating  the  use  of 
Dashboard  and  requiring  updates  for  all  ACAT  programs.  In  a  follow-up 
conversation  regarding  management  comments,  an  ASN(RD&A)  representative 
indicated  that  official  Dashboard  guidance  is  expected  to  be  issued  by  February 
2007. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 
However,  during  the  audit  follow-up  we  will  verify  that  the  ASN(RD&A)  issued 
official  guidance  to  the  Deputy  Assistant  Secretaries  of  the  Navy,  PEOs,  and 
program  managers  mandating  the  use  of  Dashboard  and  requiring  updates  for  all 
ACAT  programs. 

2,  Modify  Dashboard  to  distinguish  different  increments  for 
evolutionary  acquisition  programs. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget),  responding  for  the  ASN(RD&A),  concurred.  He 
stated  that  the  ASN(RD&A)  will  develop  the  needed  modifications  to  Dashboard 
to  allow  the  different  increments  of  evolutionary  acquisition  programs  to  be 
distinguished  and  reported.  In  a  follow-up  conversation  regarding  management 
comments,  an  ASN(RD&A)  representative  indicated  that  Dashboard  should  be 
modified  to  distinguish  different  increments  for  evolutionary  acquisition 
programs  by  the  end  of  September  2007. 

Audit  Response.  The  comments  were  responsive  to  the  recommendation. 


39 


Appendix  A.  Scope  and  Methodology 


We  evaluated  whether  the  NAE  management  oversight  and  procurement  authority 
for  ACAT  IC  and  II  programs  was  adequate.  To  accomplish  this  objective,  we 
determined  the  effectiveness  of  the  milestone  decision  authority’s  oversight  and 
whether  program  managers  provided  the  milestone  decision  authority  with 
required  program  documentation  in  support  of  program  milestone  reviews.  We 
also  evaluated  the  adequacy  of  the  procurement  process  used  by  the  Navy 
contracting  officers  to  solicit,  negotiate,  award,  and  administer  contracts. 

To  evaluate  the  adequacy  of  the  Navy  milestone  decision  authority’s  oversight  we 
judgmentally  selected  13  ACAT  IC  and  II  programs  for  review  from  a  list  of 
programs  provided  by  the  NAE.  The  NAE  was  the  milestone  decision  authority 
for  the  13  ACAT  IC  and  II  programs  selected  for  review. 

We  performed  this  audit  from  July  2005  through  October  2006,  in  accordance 
with  generally  accepted  government  auditing  standards.  We  reviewed 
documentation  dated  from  November  1988  through  October  2006.  We 
interviewed  NAE  staff.  We  interviewed  the  different  System  Commands, 
including  Naval  Sea  Systems  Command,  Washington  Navy  Yard, 

Washington,  D.C.;  Naval  Air  Systems  Command,  Patuxent  River,  Maryland;  and 
Space  and  Naval  Warfare  Command,  San  Diego,  California.  In  addition,  we 
interviewed  and  obtained  documentation  from  the  PEOs  for:  Command,  Control, 
Communications,  Computers,  and  Intelligence  and  Space;  Littoral  and  Mine 
Warfare;  Integrated  Warfare  Systems;  Air  Antisubmarine  Warfare,  Assault,  and 
Special  Mission  Programs;  Tactical  Aircraft  Programs;  and  Strike  Weapons  and 
Unmanned  Aviation.  We  interviewed  and  obtained  documentation  from  the 
program  managers  and  contracting  officers  of  the  13  programs  selected  for 
review. 

We  reviewed  documentation  in  support  of  milestone  decision  reviews  that 
include:  initial  capabilities  document;  mission  needs  statements;  operational 
requirements  documents;  capability  development  documents;  capability 
production  documents;  AoAs;  acquisition  strategies;  systems  engineering  plans; 
information  support  plans;  command,  control,  communications,  computers,  and 
intelligence  support  plans;  acquisition  plans;  test  and  evaluation  master  plans; 
affordability  assessments;  acquisition  program  baselines;  acquisition  decision 
memorandums;  program  deviation  reports;  live-fire  test  and  evaluation  reports; 
operational  evaluation  reports;  independent  cost  estimates;  beyond  low-rate  initial 
production  report;  and  milestone  decision  review  briefs. 

To  accomplish  the  audit  objectives,  we: 

•  reviewed  DoD  Directive  5000. 1 ,  “The  Defense  Acquisition  System,” 

May  12,  2003,  as  well  as  previous  applicable  DoD  acquisition  guidance, 
to  determine  DoD  policy  applicable  to  all  acquisition  programs; 

•  reviewed  DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition 
System,”  May  12,  2003,  as  well  as  previous  applicable  DoD  acquisition 
guidance,  to  determine  whether  Navy  acquisition  programs  were  assigned 


40 


the  appropriate  ACAT  and  program  managers  were  complying  with 
statutory,  regulatory,  and  contract  reporting  infonnation  and  milestone 
requirements; 

•  reviewed  SECNAV  Instruction  5000. 2C,  “Implementation  and  Operation 
of  the  Defense  Acquisition  System  and  the  Joint  Capabilities  Integration 
and  Development  System,”  November  19,  2004,  to  determine  whether  the 
NAE  implemented  the  procedures  in  DoD  Directive  5000.1  and 

DoD  Instruction  5000.2; 

•  reviewed  Federal  Acquisition  Regulation  Part  15,  “Contracting  by 
Negotiation,”  March  2005,  to  detennine  contracting  officer  requirements 
for  supporting  contract  price  reasonableness  detenninations  in  their  price 
negotiation  memorandums; 

•  determined  whether  the  conditions  identified  during  the  audits  of  the 
Boeing  KC-767A  tanker  aircraft  (DoD  IG  Report  Nos.  D-2003-129, 
“Assessment  of  DoD  Leasing  Actions,”  August  29,  2003;  and 
D-2004-064,  “Acquisition  of  the  Boeing  KC-767A  Tanker  Aircraft,” 
March  29,  2004)  and  the  C-130J  Aircraft  (DoD  IG  Report  No. 
D-2004-102,  “Contracting  for  and  Performance  of  the  C-130J  Aircraft,” 
July  23,  2004)  occurred  in  the  execution  of  Navy  acquisition  programs; 
and 

•  reviewed  the  Navy  Dashboard  Reporting  System  for  the  13  Navy 
acquisition  programs  selected  for  review  to  determine  whether  the 
required  infonnation  was  entered  into  the  Dashboard  Reporting  System 
and  whether  the  Dashboard  database  was  being  updated  timely. 

Program  Selection  and  Limitations.  We  reviewed  13  Navy  programs  that  were 
major  weapon  systems.  As  shown  in  the  following  table,  we  limited  our  review 
to  five  ACAT  IC  programs  and  eight  ACAT  II  programs  managed  by 
ASN(RD&A).  We  further  limited  our  review  by  selecting  programs  that  had  a 
recent  milestone  decision  date,  were  subject  to  DoD  5000  series  of  guidance,  and 
were  either  in  the  system  development  and  demonstration  phase  or  production  and 
deployment  phase  of  the  acquisition  process.  We  excluded  all  space  programs 
because  they  may  not  be  subject  to  the  procurement  policies  in  the  DoD  5000 
series  of  guidance.  In  addition,  we  excluded  the  following:  PEO  Submarines 
because  there  were  no  ACAT  IC  or  II  programs;  PEO  Carriers  because  they  had 
only  one  ACAT  IC  program,  which  did  not  have  a  recent  milestone  decision;  and 
PEO  Ships  because  the  only  program  that  met  our  criteria  had  been  previously 
managed  by  USD(AT&L),  who  made  all  milestone  decisions.  We  also  excluded 
programs  for  the  U.S.  Marine  Corps.  Finally,  we  did  not  visit  Defense  Contract 
Management  Agency  Offices  at  the  contractor  locations  to  review  documentation 
for  acceptance  and  delivery  of  the  systems  from  the  contractors. 
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Program  Selection  Process 


ACAT  IC 

Navy  Systems 
Commands/PEOs 

Programs 

Reviewed 

SPAWAR 

2 

1 

PEO  C4I  and  Space 

2 

1 

NAVSEA 

6 

1 

PEO  IWS 

1 

0 

PEO  LMW 

1 

1 

PEO  Ships 

2 

0 

Other* 

2 

0 

NAVAIR 

11 

3 

Air  1.0 

0 

0 

PEO  A 

3 

1 

PEOT 

3 

1 

PEO  W 

5 

1 

Total: 

19 

5 

ACAT  II 


Percentage 

Programs 

Reviewed 

Percentage 

50 

7 

1 

14 

50 

7 

1 

14 

17 

13 

4 

31 

0 

6 

2 

33 

100 

7 

2 

29 

0 

0 

0 

N/A 

0 

0 

0 

N/A 

27 

16 

3 

19 

N/A 

4 

1 

25 

33 

2 

1 

50 

33 

5 

1 

20 

20 

5 

0 

0 

26 

36 

8 

22 

SPAWAR 
NAVSEA 
PEO  C4I 

PEO  IWS 
PEO  LMW 


Space  and  Naval  Warfare  Systems  Command 
Naval  Sea  Systems  Command 

Program  Executive  Officer  Command,  Control,  Communications,  Computers, 
and  Intelligence 

Program  Executive  Officer  Integrated  Warfare  Systems 
Program  Executive  Officer  Littoral  and  Mine  Warfare 


*  Includes  two  programs  managed  by  Program  Executive  Office  Carriers  and  Strategic  Systems  Programs. 


We  also  limited  our  review  to  13  Navy  programs  that  met  certain  criteria,  such  as 
programs  that  were  managed  by  ASN(RD&A)  and  subject  to  DoD  5000  series 
guidance.  In  addition,  the  scope  of  the  audit  was  limited  in  that  we  did  not  review 
the  management  control  program. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  The  Deputy  Inspector  General  for  Policy  and 
Oversight,  Quantitative  Methods  Division  developed  the  criteria  for  selecting  the 
appropriate  judgmental  sample  size  of  Navy  ACAT  IC  and  II  programs  to  be 
reviewed. 

Government  Accountability  Office  High-Risk  Area.  The  Government 
Accountability  Office  (GAO)  has  identified  several  high-risk  areas  in  DoD.  This 
report  provides  coverage  of  the  DoD  Weapon  Systems  Acquisition  high-risk  area. 
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Appendix  B.  Prior  Coverage 


During  the  last  5  years,  GAO,  DoD  IG,  and  Naval  Audit  Service  issued  12  reports 
that  discussed  material  management  control  weaknesses  associated  with  the 
acquisition  of  weapon  systems.  Unrestricted  GAO,  DoD  IG,  and  Naval  Audit 
Service  reports  can  be  accessed  at  http://www.gao.gov,  http://www.dodig.mil/ 
audit/reports,  and  http://www.hq.navy.mil/navalaudit,  respectively. 


GAO 


GAO  Report  No.  GAO-05-301,  “Defense  Acquisitions:  Assessments  of  Selected 
Major  Weapon  Programs,”  March  2005 

GAO  Report  No.  GAO-04-393,  “Defense  Acquisitions:  Stronger  Management 
Practices  Are  Needed  to  Improve  DoD’s  Software -Intensive  Weapon 
Acquisitions,”  March  2004 

GAO  Report  No.  GAO-04-53,  “Defense  Acquisitions:  DoD’s  Revised  Policy 
Emphasizes  Best  Practices,  but  More  Controls  Are  Needed,”  November  2003 

DoD  IG 

DoD  IG  Report  No.  D-2006-001,  “Audit  of  the  Common  Submarine  Radio 
Room,”  October  3,  2005 

DoD  IG  Report  No.  D-2005-033.  “Implementation  of  Interoperability  and 
Information  Assurance  Policies  for  Acquisition  of  Navy  Systems,”  February  2, 
2005 

DoD  IG  Report  No.  D-2004-109,  “Implementation  of  the  DoD  Management 
Control  Program  for  Navy  Acquisition  Category  II  and  III  Programs,”  August  17, 
2004 

DoD  IG  Report  No.  D-2004-108,  “Implementation  of  the  DoD  Management 
Control  Program  for  Air  Force  Acquisition  Category  II  and  III  Programs,” 

August  16,  2004 

DoD  IG  Report  No.  D-2004-102,  “Contracting  for  and  Perfonnance  of  the  C-130J 
Aircraft,”  July  23,  2004 

DoD  IG  Report  No.  D-2004-064,  “Acquisition  of  the  Boeing  KC-767A  Tanker 
Aircraft,”  March  29,  2004 

DoD  IG  Report  No.  D-2004-047,  “Implementation  of  the  DoD  Management 
Control  Program  for  Anny  Acquisition  Category  II  and  III  Programs,” 

January,  23,  2004 
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DoD  IG  Report  No.  D-2003-004,  “Acquisition  of  the  Advanced  Deployable 
System,”  October  3,  2002 


Naval  Audit  Service 

Naval  Audit  Service  Report  No.  N2004-0051,  “The  AN/SPY-1D(V)  Upgrade 
Program,”  May  19,  2004 
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Appendix  C.  Glossary 


Acquisition  Category.  An  acquisition  category  is  established  to  facilitate 
decentralized  decision  making  and  execution  and  compliance  with  statutory 
requirements.  The  categories  detennine  the  level  of  review,  decision  authority, 
and  applicable  procedures. 

Acquisition  Coordination  Team.  The  Acquisition  Coordination  Team  is  a  team 
of  stakeholders  from  the  acquisition  community  who  represent  the  principal 
advisors  to  the  milestone  decision  authority.  A  team  must  be  established  for  each 
Department  of  the  Navy  ACAT  IC  and  II  program.  The  teams  are  cochaired  by 
the  cognizant  Deputy  Assistant  Secretary  of  the  Navy  or  Deputy  Assistant 
Secretary  of  the  Navy  action  officer  and  the  program  manager. 

Acquisition  Decision  Memorandum.  The  acquisition  decision  memorandum  is 
a  memorandum  signed  by  the  milestone  decision  authority.  It  documents 
decisions  made  as  the  result  of  a  milestone  decision  review  or  decision  review. 

Acquisition  Executive.  Acquisition  executives  are  the  individuals  charged  with 
the  overall  acquisition  management  responsibilities  within  their  respective 
organization. 

Acquisition  Program  Baseline.  The  acquisition  program  baseline  prescribes  the 
key  cost,  schedule,  and  performance  constraints  that  the  program  must  achieve 
before  the  next  milestone  decision  review  in  the  acquisition  process. 

Acquisition  Strategy.  An  acquisition  strategy  is  a  business  and  technical 
management  approach  designed  to  achieve  program  objectives  within  the 
resource  constraints.  It  is  the  framework  for  planning,  directing,  contracting  for, 
and  managing  a  program.  It  provides  a  master  schedule  for  research, 
development,  testing,  production,  fielding,  modification,  postproduction 
management,  and  other  activities  essential  for  program  success. 

Analysis  of  Alternatives.  An  analysis  of  alternatives  is  the  evaluation  of  the 
operational  effectiveness,  operational  suitability,  and  estimated  costs  of 
alternative  systems  to  meet  a  mission  capability.  The  analysis  assesses  the 
advantages  and  disadvantages  of  alternative  systems  being  considered  to  satisfy  a 
validated  need,  including  the  sensitivity  of  each  alternative  to  possible  changes  in 
key  assumptions  or  variables. 

Capability  Development  Document.  A  capability  development  document 
captures  the  information  necessary  to  develop  a  proposed  program,  normally 
using  an  evolutionary  acquisition  strategy.  The  document  outlines  an  affordable 
increment  of  militarily  useful,  logistically  supportable,  and  technically  mature 
capability.  It  supports  a  system  development  and  demonstration  program  decision 
review. 

Capability  Production  Document.  A  capability  production  document  addresses 
the  production  elements  specific  to  a  single  increment  of  an  acquisition  program. 
The  Deputy  Chief  of  Naval  Operations  (Resources,  Requirements,  and 
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Assessments)  must  validate  and  approve  the  document  before  a  production  and 
deployment  decision  review.  The  refinement  of  performance  attributes  and  key 
performance  parameters  is  the  most  significant  difference  between  the  capability 
development  document  and  the  capability  production  document. 

Command,  Control,  Communications,  Computers,  and  Intelligence  Support 

Plan.  The  command,  control,  communications,  computers,  and  intelligence 
support  plan  is  a  required  document  for  all  acquisition  programs  that  connect  in 
any  way  to  the  communications  and  infonnation  infrastructure,  and  includes  both 
information  technology  systems  and  National  Security  System  programs.  The 
plan  identifies  command,  control,  communications,  computers,  intelligence, 
surveillance,  and  reconnaissance  needs,  dependencies,  and  interfaces.  It  focuses 
on  interoperability,  supportability,  and  sufficiency  concerns  throughout  a 
program’s  life  cycle. 

Commander,  Operational  Test  and  Evaluation  Force.  The  Office  of 
Commander,  Operational  Test  and  Evaluation  Force  is  the  designated  Operational 
Test  Agency  for  the  United  States  Navy  and  Marine  Corps  aviation  programs 
assigned  to  Chief  of  Naval  Operations  sponsorship.  The  Commander, 

Operational  Test  and  Evaluation  Force  plans,  conducts,  evaluates,  and  reports  the 
operational  test  and  evaluation  of  AC  AT  I  and  II  programs. 

Developmental  Test  and  Evaluation.  Developmental  test  and  evaluation  is  any 
engineering  test  used  to  verify  status  of  weapon  systems  technical  progress,  verify 
that  design  risks  are  minimized,  substantiate  achievement  of  contract  technical 
performance,  and  certify  readiness  for  initial  operational  testing.  Development 
tests  generally  require  instrumentation  and  measurements  and  are  accomplished 
by  engineers,  technicians,  or  soldier  operator-maintainer  test  personnel  in  a 
controlled  environment  to  facilitate  analysis  of  failures. 

Director  Test  and  Evaluation  Oversight  List.  The  annual  Office  of  the 
Secretary  of  Defense  Test  and  Evaluation  Oversight  List  identifies  those 
Department  of  the  Navy  programs  subject  to  Office  of  the  Secretary  of  Defense 
Test  and  Evaluation  oversight.  ACAT  I,  ACAT  II,  and  programs  requiring 
live-fire  test  and  evaluation  are  generally  included  in  oversight. 

Engineering  Development  Model.  An  engineering  development  model  is  a 
production  representative  system  acquired  during  the  system  development  and 
demonstration  phase  of  the  acquisition  process.  Engineering  development  models 
may  be  used  to  demonstrate  maturing  performance  via  an  operational  assessment 
or  operational  testing. 

Evolutionary  Acquisition  Programs.  Evolutionary  acquisition  is  the  preferred 
DoD  strategy  for  rapid  acquisition  of  mature  technology  for  the  user.  An 
evolutionary  approach  delivers  capability  in  increments,  recognizing,  up  front,  the 
need  for  future  capability  improvements.  The  objective  is  to  balance  needs  and 
available  capability  with  resources  and  to  put  capability  into  the  hands  of  the  user 
quickly. 

Full-Rate  Production.  Full-rate  production  represents  contracting  for  economic 
production  quantities  following  stabilization  of  the  system  design  and  validation 
of  the  production  process. 
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Information  Support  Plan.  The  information  support  plan  (formerly  known  as 
the  command,  control,  communications,  computers,  and  intelligence  support  plan) 
identifies  and  documents  information  technology  and  national  security  system 
needs,  objectives,  and  interface  requirements  in  sufficient  detail  to  enable  testing 
and  verification  of  requirements.  The  infonnation  support  plan  also  contains 
interface  descriptions,  infrastructure  and  support  requirements,  standards  profiles, 
measures  of  perfonnance,  and  interoperability  shortfalls. 

Inventory  Objective.  Inventory  objective  is  the  quantity  of  an  item  of  material 
that  will  satisfy  the  military  requirement  under  specified  mobilization  conditions. 

Key  Performance  Parameter.  Key  performance  parameters  are  those  minimum 
attributes  or  characteristics  considered  most  essential  for  an  effective  military 
capability.  For  capabilities  documents,  key  performance  parameters  are  validated 
by  the  Joint  Requirements  Oversight  Council  for  joint  requirements  documents, 
by  the  Functional  Capabilities  Board  for  requirements  documents  that  jointly 
affect  Services,  and  by  the  DoD  Component  for  requirements  documents  that  are 
prepared  independently  by  a  Service. 

Low-Rate  Initial  Production.  Low-rate  initial  production  is  the  first  segment  of 
the  production  and  deployment  phase.  Its  purpose  is  to  establish  an  initial 
production  base  for  the  system,  permit  an  orderly  ramp-up  sufficient  to  lead  to  a 
smooth  transition  to  full-rate  production,  and  to  provide  production-representative 
articles  for  initial  operational  test  and  evaluation  and  live-fire  testing.  For  major 
Defense  acquisition  programs,  low-rate  initial  production  quantities  in  excess  of 
10  percent  of  the  acquisition  objective  must  be  reported  in  the  Selected 
Acquisition  Report. 

Milestone  Decision  Authority.  The  milestone  decision  authority  is  the 
designated  individual  with  overall  responsibility  for  a  program.  The  milestone 
decision  authority  approves  program  initiation  and  entry  of  an  acquisition 
program  into  the  next  phase  of  the  acquisition  process.  The  milestone  decision 
authority  is  accountable  for  cost,  schedule,  and  performance  reporting  to  higher 
authority,  including  congressional  reporting. 

Operational  Effectiveness.  Operational  effectiveness  is  a  measure  of  the  overall 
ability  of  a  system  to  accomplish  a  mission  when  used  by  representative 
personnel  in  the  environment  planned  or  expected  for  operational  employment  of 
the  system. 

Operational  Requirements  Document.  The  operational  requirements  document 
is  a  legacy  document  that  contains  performance  parameters  for  the  proposed 
concept  or  system.  Operational  requirements  documents  were  accepted  for  Joint 
Staff  review  until  late  December  2003.  Capability  development  documents  and 
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capability  production  documents  developed  in  accordance  with  Chainnan  of  the 
Joint  Chiefs  of  Staff  Instruction  3 170.0 1C,  “Joint  Capabilities  Integration  and 
Development  System,”  June  24,  2003,  replaced  the  operational  requirements 
document.  The  instruction  allowed  a  validated  and  approved  operational 
requirements  document  to  be  used  to  support  program  initiation  and  the  low-rate 
initial  production  decision  until  late  June  2005. 

Operational  Suitability.  Operational  suitability  is  the  degree  to  which  a  system 
can  be  placed  and  sustained  satisfactorily  in  field  use. 

Operational  Test  and  Evaluation.  Operational  test  and  evaluation  is  the  field 
test,  under  realistic  conditions,  of  any  item  (or  key  component)  of  weapons, 
equipment,  or  munitions  to  determine  its  effectiveness  and  suitability  for  use  in 
combat  by  typical  military  users. 

Program  Executive  Officer.  A  program  executive  officer  is  a  military  or 
civilian  official  who  has  the  responsibility  for  directing  several  major  Defense 
acquisition  programs  and  for  directing  major  and  nonmajor  system  acquisition 
programs.  A  program  executive  officer  has  no  other  command  or  staff 
responsibilities  within  the  Component,  and  reports  to  and  receives  guidance  and 
direction  from  only  the  DoD  Component  Acquisition  Executive. 

Program  Manager.  The  program  manager  is  a  designated  individual  with 
responsibility  for  and  authority  to  accomplish  program  objectives  for 
development,  production,  and  sustainment  to  meet  the  users’  operational  needs. 
The  program  manager  is  accountable  for  cost,  schedule,  and  performance 
reporting  to  the  milestone  decision  authority. 

Research,  Development,  Test,  and  Evaluation.  Research,  development,  test, 
and  evaluation  are  activities  for  developing  a  new  system  or  expanding  the 
performance  of  fielded  systems. 

System  Development  and  Demonstration.  The  system  development  and 
demonstration  phase  (Milestone  B)  is  the  third  phase  of  the  DoD  system 
acquisition  process  and  consists  of  system  integration  and  system  demonstration. 
This  phase  also  contains  a  design  readiness  review  at  the  conclusion  of  the  system 
integration. 

Systems  Engineering  Plan.  The  systems  engineering  plan  describes  the 
program’s  overall  technical  approach,  including  processes,  resources,  metrics,  and 
applicable  perfonnance  incentives.  It  also  states  the  timing,  conduct,  and  success 
criteria  of  technical  reviews. 

Test  and  Evaluation  Master  Plan.  The  test  and  evaluation  master  plan 
documents  the  overall  structure  and  objectives  of  the  test  and  evaluation  program. 
It  provides  a  framework  for  generating  detailed  test  and  evaluation  plans  and  for 
documenting  schedule  and  resource  implications  associated  with  the  test  and 
evaluation  program.  The  test  and  evaluation  master  plan  identifies  the  necessary 
developmental  test  and  evaluation,  operational  test  and  evaluation,  and  live-fire 
test  and  evaluation  activities. 
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Test  Report.  A  test  report  formally  documents  the  results,  conclusions,  and 
recommendations  that  result  from  each  phase  of  developmental  testing  and 
operational  testing. 

Threshold.  A  threshold  is  a  minimum  acceptable  operational  value  below  which 
the  utility  of  the  system  becomes  questionable. 
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Appendix  D.  Comparison  of  Conditions  Identified 

on  Audits  of  Boeing  KC-767A  and 
C-130J  Aircraft 


As  a  result  of  audits  of  the  Boeing  KC-767A  tanker  aircraft  (DoD  IG  Report  Nos. 
D-2003-129,  “Assessment  of  DoD  Leasing  Actions,”  August  29,  2003;  and 
D-2004-064,  “Acquisition  of  the  Boeing  KC-767A  Tanker  Aircraft,”  March  29, 
2004)  and  the  C-130J  aircraft  (DoD  IG  Report  No.  D-2004-102,  “Contracting  for 
and  Perfonnance  of  the  C-130J  Aircraft,”  July  23,  2004),  the  DoD  IG  initiated  the 
series  of  audits  of  the  Service  Acquisition  Executives  to  detennine  whether 
management  oversight  problems  identified  in  the  referenced  reports  were  more 
widely  occurring  across  the  Military  Departments.  At  the  April  14,  2005,  hearing 
on  management  and  oversight  of  Air  Force  acquisition  programs,  the  Senate 
Anned  Services  Subcommittee  also  expressed  interest  in  the  results  of  the 
DoD  IG  audits  concerning  whether  similar  conditions  were  occurring  within  the 
other  Services.  The  results  of  the  review  of  13  Navy  AC  AT  IC  and  II  programs 
as  related  to  the  15  conditions  identified  in  the  earlier  audits  of  the  two  Air  Force 
acquisition  programs  follows. 

1.  Condition:  The  former  Deputy  Assistant  Secretary  of  the  Air  Force 
(Acquisition)  used  her  position  as  the  milestone  decision  authority  and  head  of 
Air  Force  contracting  to  conduct  and  inappropriately  influence  the  results  of  the 
contract  negotiations  with  Boeing  to  acquire  Boeing  KC-767A  tanker  aircraft. 

Question:  Did  the  NAEs  use  their  positions  as  milestone  decision  authorities  to 
conduct  and  inappropriately  influence  the  results  of  contractor  selection  and 
negotiations  for  the  13  programs  selected  for  review? 

Results:  For  the  13  programs  reviewed,  we  did  not  find  evidence  that  the  NAEs 
used  their  positions  to  inappropriately  influence  the  results  of  contractor  selection 
and  negotiations. 

2,  Condition:  On  both  the  Boeing  KC-767A  tanker  aircraft  and  the  C-130J 
programs,  Air  Force  contracting  officers  did  not  properly  justify  the  use  of  a 
commercial  item  acquisition  strategy.  The  Federal  Acquisition  Regulation  (FAR) 
states  that  a  commercial  item  is  any  item,  other  than  real  property,  that  is  used 
customarily  by  the  general  public  or  by  nongovernmental  entities  for  purposes 
other  than  governmental  purposes.  Further,  commercial  items  are  those  that  have 
been  sold,  leased,  or  licensed  to  the  general  public;  or  have  been  offered  for  sale, 
lease,  or  license  to  the  general  public. 

Question:  Did  Navy  contracting  officers  use  and  properly  justify  the  use  of  a 
commercial  item  acquisition  strategy  for  the  13  programs  selected  for  review? 

Results:  For  the  13  programs  reviewed,  Navy  contracting  officers  did  not  use  a 
commercial  item  acquisition  strategy.  Navy  contracting  officers  included  FAR 
Subpart  15,  “Contracting  by  Negotiation,”  as  part  of  their  acquisition  planning. 


50 


3.  Condition:  On  the  C-130J  program,  the  Air  Force  conditionally  accepted  the 
delivery  of  C-130J  aircraft  that  did  not  meet  commercial  contract  specifications  or 
operational  requirements. 

Question:  Did  the  Navy  acquisition  officials  conditionally  accept  delivery  of 
items  before  the  items  met  contract  specifications  and  operational  requirements 
for  the  13  programs  selected  for  review? 

Results:  As  mentioned  in  Appendix  A,  we  did  not  visit  Defense  Contract 
Management  Agency  offices  at  the  contractor  locations  or  otherwise  determine 
whether  the  systems  for  the  13  programs  reviewed  had  been  accepted  before 
meeting  contract  specifications  or  operational  requirements. 

4.  Condition:  On  the  C-130J  program,  the  Air  Force  contracting  officer  did  not 
adequately  manage  the  financing  of  the  contract.  This  inadequate  management 
resulted  in  the  Air  Force  paying  the  contractor  85  percent  of  the  price  of  the 
aircraft  before  aircraft  acceptance  inspection  and  99  percent  of  the  price  of  the 
aircraft  on  conditional  acceptance  and  delivery  of  noncompliant  aircraft. 

Question:  Did  Navy  contracting  officers  properly  manage  the  financing  of 
deliverable  items  on  contracts  for  the  13  programs  selected  for  review? 

Results:  On  5  of  1 1  production  contracts,  the  Navy  contracting  officers  included 
FAR  Clause  52.232-16,  “Progress  Payments,”  or  Defense  Federal  Acquisition 
Regulation  Supplement  Clause  252.232-7004,  “DoD  Progress  Payment  Rates,” 
which  limited  the  contractor  financing  to  80  percent  or  less  of  the  contract  price 
before  acceptance  of  the  deliverable  items.  The  remaining  six  programs  did  not 
include  either  clause  for  progress  payments,  and  we  did  not  detennine  the  manner 
in  which  financing  occurred. 

5.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  program,  the  Air  Force 
contracting  officer  negotiated  a  prohibited  cost-plus-a-percentage-of-cost 
contract.  Cost-plus-a-percentage-of-cost  contracts  are  prohibited  by  Section 
2306(a),  title  10,  United  States  Code.  A  cost-plus-a-percentage-of-cost  contract  is 
a  cost  reimbursement  contract  that  provides  a  contractor  a  fee  based  as  a  specified 
percentage  of  the  contractor’s  actual  cost  of  perfonning  the  work.  According  to 
GAO,  a  cost-plus-a-percentage-of-cost  contract  occurs  when  contracting  officers 
award  a  contract  for  which: 

•  payment  for  profit  is  based  on  a  predetermined  percentage  rate, 

•  a  predetennined  percentage  rate  applies  to  the  actual  cost  of  work 
performed, 

•  contractor  entitlement  is  uncertain  at  the  time  of  contracting, 

•  contractor  entitlement  increases  commensurately  with  increased 
performance  costs,  and 

•  Government  audit  rights  are  excluded. 
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Question:  Did  Navy  contracting  officers  use  a  prohibited  cost-plus-a-percentage- 
of-cost  system  of  contracting  through  the  use  of  limitation  of  earnings  clauses  and 
the  exclusion  of  Government  audit  rights  on  the  13  programs  selected  for  review? 

Results:  We  did  not  identify  any  instances  where  the  Navy  contracting  officers 
structured  and  awarded  a  cost-plus-a-percentage-of-cost  contract.  We  reviewed 
the  basic  contract  clauses  and  the  business  clearance  memorandums  for  the 
13  Navy  programs.  None  of  the  13  acquisition  programs  in  our  review  used  a 
cost-plus-a-percentage-of-cost  contract.  All  13  contracts  we  reviewed  included 
the  Government  audit  rights  clause  as  part  of  contract. 

6.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  program,  the  proposed 
lease  did  not  meet  all  of  the  criteria  requirements  for  an  operating  lease.  Further, 
the  proposed  lease  would  have  cost  the  Air  Force  more  than  purchasing  the 
aircraft.  Office  of  Management  and  Budget  Circular  A-l  1,  “Preparation, 
Submission,  and  Execution  of  the  Budget,”  June  2006,  states  that  an  operating 
lease  must  meet  the  following  six  requirements: 

•  The  asset  is  a  general-purpose  asset  rather  than  being  for  a  special  purpose 
of  the  Government  and  is  not  built  to  unique  specification  of  the 
Government  as  a  lessee. 

•  There  is  a  private-sector  market  for  the  asset. 

•  The  present  value  of  the  minimum  lease  payments  over  the  life  of  the 
lease  does  not  exceed  90  percent  of  the  fair  market  value  of  the  asset  at  the 
beginning  of  the  lease  term. 

•  The  lease  does  not  contain  a  bargain-price  purchase  option. 

•  Ownership  of  the  asset  remains  with  the  lessor  during  the  term  of  the  lease 
and  is  not  transferred  to  the  Government  at  or  shortly  after  the  end  of  the 
lease  tenn. 

•  The  lease  tenn  does  not  exceed  75  percent  of  the  estimated  economic  life 
of  the  asset. 

Question:  Did  Navy  contracting  officers  use  and  properly  justify  the  use  of 
leases  in  accordance  with  Office  of  Management  and  Budget  Circular  A-l  1  on  the 
13  programs  selected  for  review? 

Results:  The  Navy  contracting  officers  did  not  use  leases  for  the  13  programs 
reviewed. 

7.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  program,  the  Air  Force 
contracting  officer  did  not  require  Boeing  to  submit  cost  and  pricing  data  related 
to  prior  commercial  sales  to  enable  the  Air  Force  contracting  officer  to  detennine 
price  reasonableness. 

Question:  Did  Navy  contracting  officers  require  contractors  to  submit  cost  or 
pricing  data  to  enable  the  contracting  officers  to  determine  price  reasonableness 
for  the  13  programs  selected  for  review? 
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Results:  As  indicated  in  price  negotiation  memorandums  for  8  of  13  programs 
reviewed.  Navy  contracting  officers  required  and  relied  on  cost  or  pricing  data  to 
negotiate  the  contract  price  and  support  a  price  reasonableness  determination. 
Three  of  the  13  were  exempt  from  using  cost  or  pricing  data  and  1  program  did 
not  require  cost  and  pricing  data  due  to  having  significant  historical  cost  and 
pricing  data.  The  remaining  program  planned  to  use  cost  or  pricing  data  when  the 
contract  was  definitizcd. 

8.  Condition:  The  Assessment  of  Leasing  Actions  report  stated  that  the 

Air  Force  took  full  advantage  of  Section  8159  of  the  DoD  Appropriations  Act  for 
FY  2002  that  authorized  the  Air  Force  to  lease  not  more  than  100  general  purpose 
Boeing  KC-767A  aircraft.  With  this  authority,  the  Air  Force  did  not  prepare  a 
formal  analysis  of  alternatives  to  determine  the  best  possible  system  solution  to 
fulfill  its  need  for  a  tanker  aircraft  replacement.  DoD  Instruction  5000.2  requires 
that  an  analysis  of  alternatives  be  completed  and  approved  by  the  Director, 
Program  Analysis  and  Evaluation  for  major  Defense  acquisition  programs.  The 
analysis  of  alternatives  is  an  evaluation  of  the  system’s  operational  effectiveness, 
operational  suitability,  and  estimated  costs  of  alternative  systems  to  meet  a 
mission  capability.  The  analysis  assesses  the  advantages  and  disadvantages  of 
alternatives  being  considered  to  satisfy  capabilities,  including  the  sensitivity  of 
each  alternative  to  possible  changes  in  key  assumptions  or  variables. 

Question:  Did  the  Navy  prepare  an  analysis  of  alternatives  to  support  the 
acquisition  of  the  13  programs  selected  for  review? 

Results:  Of  the  13  programs  reviewed,  7  had  completed  an  analysis  of 
alternatives  or  cost  and  operational  effectiveness  analysis  in  accordance  with  DoD 
Instruction  5000.2  and  Secretary  of  the  Navy  Instruction  5000.2C.  Of  the 
remaining  six  programs,  one  program  had  an  analysis  of  alternatives  or  cost  and 
operational  effectiveness  analysis  but  did  not  have  signature  pages  to  substantiate 
that  the  documents  had  been  approved  by  the  NAE.  One  program  had  an  analysis 
of  alternatives  but  it  was  not  properly  approved.  Program  officials  for  four 
programs  were  unable  to  provide  their  analyses  of  alternatives  or  cost  and 
operational  effectiveness  analyses. 

9.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  program,  the  KC-767A 
System  Program  Office  did  not  establish  a  disciplined  acquisition  strategy  to 
ensure  that  the  warfighters’  operational  requirements  were  being  satisfied.  The 
Office  of  Management  and  Budget  Circular  A- 109,  “Major  System  Acquisitions,” 
April  5,  1976,  states  that  Federal  agencies  should  tailor  an  acquisition  strategy  for 
each  major  system  to  ensure  that  each  major  system  fulfills  a  mission  need  and 
operates  effectively  in  its  intended  environment.  DoD  Instruction  5000.2  requires 
the  program  manager  to  prepare  and  the  milestone  decision  authority  to  approve 
an  acquisition  strategy  by  the  system  development  and  demonstration  phase  of  the 
acquisition  process. 

Question:  Did  Navy  program  managers  prepare  acquisition  strategies  in 
accordance  with  requirements  in  Office  of  Management  and  Budget 
Circular  A- 109  and  DoD  Instruction  5000.2  for  the  13  programs  selected  for 
review? 
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Results:  Of  the  13  programs  reviewed,  9  had  completed  acquisition  strategies  in 
accordance  with  DoD  Instruction  5000.2.  As  discussed  in  finding  A,  program 
managers  for  the  other  four  programs  either  did  not  fulfill  the  requirements  for  the 
acquisition  strategy  as  listed  in  DoD  Instruction  5000.2  or  did  not  provide  the 
auditors  with  their  program’s  acquisition  strategy. 

10.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  program,  the  program 
manager  did  not  plan  to  complete  an  information  support  plan  (formerly  referred 
to  as  a  command,  control,  communications,  computers,  and  intelligence  support 
plan)  before  the  milestone  decision  to  acquire  the  first  100  tanker  aircraft.  An 
information  support  plan  is  needed  to  identify,  plan,  and  manage  command, 
control,  communications,  computers,  and  intelligence  supportability  needs; 
dependencies  between  systems;  and  interface  and  interoperability  requirements. 
DoD  Instruction  5000.2  requires  program  managers  to  prepare  an  information 
support  plan  before  the  decision  reviews  for  entering  into  the  system  development 
and  demonstration  and  the  production  and  deployment  phases  of  the  acquisition 
process. 

Question:  Did  the  Navy  milestone  decision  authority  require  program  managers 
to  prepare  and  obtain  approval  for  infonnation  support  plans  before  the  system 
development  and  demonstration  and  production  and  deployment  phases  of  the 
acquisition  process  for  the  13  programs  selected  for  review? 

Results:  Program  officials  for  8  of  the  13  programs  had  completed  an 
information  support  plan  or  a  command,  control,  communications,  computers,  and 
intelligence  support  plan  and  obtained  approval  before  the  system  development 
and  demonstration  or  production  and  deployment  phases  of  the  acquisition 
process.  For  three  of  the  remaining  programs,  one  received  a  waiver  to  the 
development  of  the  infonnation  support  plan,  and  two  followed  the  guidance 
provided  to  them  by  the  Office  of  the  Assistant  Secretary  of  Defense  (Networks 
and  Infonnation  Integration)/ Chief  Information  Officer.  As  discussed  in  finding 
A,  the  remaining  two  programs  did  not  fulfill  the  requirements  for  preparing  an 
information  support  plan  or  command,  control,  communications,  computers,  and 
intelligence  support  plan. 

11.  Condition:  The  operational  requirements  document  developed  by  the 
Air  Force  did  not  require  that  the  first  100  Boeing  KC-767A  tanker  aircraft 
acquired  meet  warfighter  requirements  for  interoperability.  As  a  result,  the 
aircraft  acquired  would  not  have  fully  met  the  key  performance  parameter  for 
interoperability. 

Question:  Did  the  NAE  require  program  managers  to  meet  a  net-ready  (fonnerly 
interoperability)  key  performance  parameter  in  the  capability  development 
documents  and  capability  production  documents? 

Results:  Of  the  13  programs  reviewed,  12  met  the  requirement  to  have  key 
performance  parameters  for  net-readiness  or  interoperability  in  the  capability 
development  documents  and  capability  production  documents.  One  of  the 
13  programs  did  not  have  key  performance  parameters  for  net-readiness  or 
interoperability  because  the  program  did  not  have  a  requirements  document  (see 
finding  A). 
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12.  Condition:  The  Air  Force  did  not  ensure  that  warfighter  operational 
requirements  were  adequately  established  in  the  contract  specifications  for  the 
Boeing  KC-767A  tanker  program.  The  Air  Force  also  accepted  C-130J  aircraft 
that  did  not  meet  contract  specifications  and  therefore  could  not  perfonn  its 
operation  mission. 

Question:  Did  Navy  program  managers  ensure  that  contracting  officers  included 
the  requirements  identified  in  the  operational  requirements  or  capabilities 
development  documents  in  contract  specification  before  awarding  development 
contracts  for  the  five  Navy  weapon  systems  reviewed  in  the  system  development 
and  demonstration  phase  of  the  acquisition  process? 

Results:  For  four  of  the  five  Navy  programs  that  were  in  the  system  development 
and  demonstration  phase  of  the  acquisition  process,  program  managers  had 
verified  that  contracting  officers  included  requirements  identified  in  operational 
requirements  documents  or  capabilities  development  documents  in  system 
contract  specifications.  However,  for  the  Navy  Advanced  Extremely  High 
Frequency  Multiband  Terminal,  we  were  unable  to  verify  whether  the  contractors 
included  the  requirements  identified  in  the  operational  requirements  documents  in 
the  system  contract  specifications  because  the  program  officials  did  not  respond 
to  our  requests  for  information. 

13.  Condition:  The  Air  Force  did  not  comply  with  statutory  provisions  for 
determining  the  operational  effectiveness,  suitability,  and  survivability  of  the 
Boeing  KC-767A  tanker  aircraft  before  proceeding  beyond  low-rate  initial 
production  and  committing  to  the  subsequent  production  of  all  100  KC-767A 
tanker  aircraft.  Section  2399,  title  10,  United  States  Code  states  that  a  major 
Defense  acquisition  program  may  not  proceed  beyond  low-rate  initial  production 
until  initial  operational  test  and  evaluation  of  the  program  is  completed.  Further, 
section  2366,  title  10,  United  States  Code  states  that  a  covered  system  (a  system 
under  the  oversight  of  the  Director,  Operational  Test  and  Evaluation)  may  not 
proceed  beyond  low-rate  initial  production  until  realistic  survivability  testing  of 
the  system  has  been  completed. 

Question:  Did  the  Navy  milestone  decision  authorities  ensure  that  initial 
operational  test  and  evaluation  was  completed  before  they  approved  the  four 
Navy  acquisition  programs  for  full-rate  production?  Also,  did  the  milestone 
decision  authority  ensure  that  survivability  testing  was  planned  and  conducted  for 
the  two  covered  acquisition  programs? 

Results:  The  Navy  milestone  decision  authorities  ensured  that  program  managers 
for  the  four  acquisition  programs  in  production  completed  initial  operation  test 
and  evaluation  before  approving  the  programs  for  full-rate  production.  Further, 
Navy  milestone  decision  authorities  ensured  that  program  managers  completed 
survivability  testing  for  two  of  four  systems  before  approving  the  acquisition 
programs  for  full-rate  production.  Two  of  the  four  systems  were  not  covered; 
therefore,  the  Director,  Operational  Test  and  Evaluation  did  not  require  program 
officials  to  conduct  survivability  testing. 

14.  Condition:  Costly  contract  modifications  to  convert  the  commercial  aircraft 
to  the  KC-767A  military  configuration  would  occur  because  the  KC-767A 
System  Program  Office  did  not  fully  develop  systems  engineering  requirements. 
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Question:  Did  Navy  program  managers  prepare  systems  engineering  plans  for 
the  13  Navy  programs  reviewed? 

Results:  According  to  the  Under  Secretary  of  Defense  (Acquisition,  Technology, 
and  Logistics)  memorandum,  “Policy  for  Systems  Engineering  in  DoD,” 

February  20,  2004,  8  of  the  13  programs  should  have  developed  a  systems 
engineering  plan.  Six  of  eight  programs  had  a  systems  engineering  plan  in 
accordance  with  the  memorandum  while  two  of  the  eight  programs  did  not  have 
properly  approved  systems  engineering  plans.  The  memorandum  did  not  apply  to 
the  remaining  five  programs  because  these  programs  had  not  had  a  milestone 
decision  review  since  the  memorandum  was  issued  or  were  already  in  low-rate 
initial  production  or  beyond  when  the  memorandum  was  issued. 

15.  Condition:  On  the  Boeing  KC-767A  tanker  aircraft  and  the  C-130J 
programs,  the  Assistant  Secretary  of  the  Air  Force  (Acquisition)  did  not  hold 
program  managers  accountable  for  completing  statutory  and  regulatory 
requirements.  The  DoD  IG  Report  No.  D-2004-064,  “Acquisition  of  the  Boeing 
KC-767A  Tanker  Aircraft,”  March  29,  2004,  cited  requirements  in  the  areas  of 
commercial  items;  two  statutory  testing  requirements;  cost-plus-a-percentage-of- 
cost  system  of  contracting;  leases;  and  acquisition  documentation,  such  as  the 
acquisition  strategy  and  requirements  documents.  DoD  IG  Report  No. 
D-2004-102,  “Contracting  for  and  Performance  of  the  C-130J  Aircraft,”  July  23, 
2004,  cited  requirements  in  the  areas  of  commercial  items,  multiyear  contract 
award,  and  testing. 

Question:  Did  the  Navy  milestone  decision  authority  hold  program  managers 
accountable  for  completing  statutory  and  regulatory  document  requirements 
before  milestone  decisions  and  program  reviews  for  the  13  programs  selected  for 
review? 

Results:  Three  of  13  programs  reviewed  had  completed  statutory  and  regulatory 
documentation  requirements  in  accordance  with  DoD  Instruction  5000.2.  As 
discussed  in  finding  A,  10  of  the  13  programs  reviewed  either  did  not  fulfill  the 
requirements  for  the  documentation  as  listed  in  DoD  Instruction  5000.2  or  could 
not  provide  the  auditors  with  the  programs’  relevant  documentation. 
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Appendix  E.  Overview  of  the  13  Navy  Programs 


Program  Name 

Phase 

Last  Milestone 

Decision 

ACAT 

Advanced  Deployable  System 

SD&D 

December  22,  2005 

IC 

AN/APG-79  Active  Electronically  Scanned 
Array  Radar  System 

P&D/LRIP  IV 

Jan.  29,  2004/Feb.  28,  2006 

IC 

MH-60R  Multi-Mission  Helicopter 

P&D/FRP 

March  31,  2006 

IC 

Navy  Advanced  Extremely  High  Frequency 
Multiband  T erminal 

SD&D 

October  21,  2003 

IC 

Tactical  Tomahawk 

P&D/FRP 

August  3,  2004 

IC 

Advanced  Arresting  Gear 

SD&D 

February  10,  2005 

II 

AN/AES- 1  Airborne  Laser  Mine  Detection 
System 

P&D/LRIP 

June  14,  2005 

II 

AN/AQS-20A  Sonar  Mine  Detecting  Set 

P&D/LRIP 

May  10,  2005 

II 

AN/SPY- 1D(V)  Radar  Upgrade 

LRIP  II1 

Sept.  29,  1993/Dec.  21,  2001 

II 

AV-8B  Harrier  II  Open  Systems  Core 
Avionics  Requirements 

P&D/FRP 

August  16,  2004 

II 

Common  Link  Integration  Processing 

SD&D2 

June  1,  2005 

II 

Mark  XIIA  Mode  5  Identification  Friend  or 
Foe  Upgrade 

SD&D 

August  7,  2003 

II 

Surface  Electronic  Warfare  Improvement 
Program 

P&D/LRIP3 

December  23,  2005 

II 

DOT&E 

Director,  Operational  Test  and  Evaluation 

FRP 

Full-Rate  Production 

GWOT 

Global  War  on  Terrorism 

LRIP 

Low-Rate  Initial  Production 

P&D 

Production  and  Deployment 

RDT&E 

Research,  Development,  Test,  and  Evaluation 

SD&D 

System  Development  and  Demonstration  Phase 

RDT&E 
(in  millions) 

Procurement 
(in  millions) 

Total  Cost 
(in  millions) 

GWOT 
Funding 
(in  millions) 

DOT&E 

Oversight 

Urgent 

Need 

$788.33 

$667.29 

$1,455.62 

0 

Yes 

No 

641.34 

2,057.75 

2,699.09 

0 

Yes 

No 

1,670.9 

10,018.8 

11,689.70 

0 

Yes 

No 

407.15 

1166.2 

1,573.35 

0 

Yes 

No 

756.8 

4,151.78 

4,908.58 

$75.9 

Yes 

No 

196.56 

601.5 

798.06 

0 

No 

No 

115.58 

327.61 

443.19 

0 

No 

No 

210.32 

1612.36 

1,822.68 

0 

No 

No 

290.24 

5983.15 

6,273.39 

0 

Yes 

No 

351.65 

262.28 

613.93 

0 

No 

No 

330.58 

0 

330.58 

0 

No 

No 

253.20 

628.31 

881.51 

0 

Yes 

No 

156.83 

766.94 

923.77 

0 

Yes 

No 

$6,169.5 

$28,244.0 

$34,413.5 

$75.9 

Note:  All  funding  figures  converted  to  FY  2006  dollars  based  on  program’s  base  year  amounts 


1  At  the  time  of  the  last  milestone  for  the  AN/SPY- 1  Radar  Upgrade  program,  September  29,  1993,  LRIP  was  not  yet  considered  a  P&D  phase. 

2  Common  Link  Integration  Processing  program  is  an  evolutionary  acquisition  program.  Increment  1  is  currently  in  the  SD&D  phase. 

3  Surface  Electronic  Warfare  Improvement  Program  is  an  evolutionary  acquisition  program.  The  two  subincrements  of  Block  1A  are  currently  in  the  P&D/LRIP  phase. 
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Appendix  F.  Description  of  the  13  Navy  Programs 


Acquisition  Category  IC  Programs 


Advanced  Deployable  System.  The  Advanced  Deployable  System  program  is 
under  the  PEO  for  Littoral  and  Mine  Warfare.  It  is  a  rapidly  deployable  passive 
acoustic  undersea  surveillance  system  designed  to  detect  and  track  modern  diesel- 
electric  submarines  and  surface  craft  in  the  shallow  waters  of  the  littoral  regions 
of  the  world.  The  NAE  approved  the  program  for  system  development  and 
demonstration  on  December  22,  2005. 

AN/APG-79  Active  Electronically  Scanned  Array  Radar  System.  The 

AN/APG-79  Active  Electronically  Scanned  Array  Radar  System  program  is  under 
the  PEO  for  Tactical  Aircraft  Programs.  It  is  the  primary  search,  track,  and 
weapons  control  radar  for  the  F/A-18E/F  aircraft.  The  system’s  antenna  is  an 
electronically  scanned  antenna  composed  of  many  active  transmitting  and 
receiving  elements.  A  computer  controls  the  antenna  elements  individually,  or  in 
groups,  to  electronically  steer  a  radar  beam  for  various  tactical  purposes.  The 
NAE  approved  the  program  low-rate  initial  production  phase  on  July  21,  2003. 
The  system  was  approved  for  a  fourth  low-rate  initial  production  on 
February  28,  2006. 

MH-60R  Multi-Mission  Helicopter.  The  MH-60R  program  is  under  the  PEO 
for  Air  Antisubmarine  Warfare,  Assault,  and  Special  Mission  Programs.  The 
program  is  an  upgrade  that  provides  critical  improvements  to  an  existing  aircraft. 
The  SH-60B  and  SH-60F  aircraft  will  be  replaced  with  a  new  air  vehicle  and 
upgraded  mission  avionics  in  a  configuration  designated  as  the  MH-60R  Weapon 
System.  The  NAE  approved  the  program  for  full-rate  production  phase  on 
March  31,  2006. 

Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal.  The  Navy 
Advanced  Extremely  High  Frequency  Multiband  Terminal  program  is  under  the 
PEO  for  Command,  Control,  Communications,  Computers,  and  Intelligence  and 
Space.  The  program  is  the  next  generation  Navy  Extremely  High  Frequency 
tenninal.  It  will  provide  deployed  Naval  commanders  with  secure  command  and 
control  capability  via  Advanced  Extremely  High  Frequency  satellites,  as  well  as 
other  military  and  commercial  satellites  from  anywhere  in  the  world.  The  NAE 
approved  the  program  for  system  development  and  demonstration  on 
October  21,  2003. 

Tactical  Tomahawk.  The  Tactical  Tomahawk  program  is  under  the  PEO  for 
Strike  Weapons  and  Unmanned  Aviation.  The  Tactical  Tomahawk  program 
replaces  the  Tomahawk  Baseline  Improvement  Program  by  providing  a  more 
responsive,  flexible,  and  lower  cost  missile  to  meet  the  evolving  tactical  scenarios 
of  the  future.  Block  IV  provides  significantly  improved  navigational  accuracy  in 
the  Global  Positioning  System  jamming  environment,  unique  in-flight  retargeting 
capability,  and  the  transmission  of  battle  damage  indication  imagery  through  a 
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two-way  satellite  data  link.  The  NAE  approved  the  program  for  full-rate 
production  on  August  3,  2004. 


Acquisition  Category  II  Programs 


Advanced  Arresting  Gear.  The  Advanced  Arresting  Gear  program  is  under  the 
Assistant  Commander  for  Acquisition,  Air  1.0.  The  program  is  a  new  aircraft 
carrier  arresting  gear  system  to  replace  the  MK  7  Mod  3  arresting  gear.  It  will 
provide  new  operational  capabilities,  including  the  ability  to  safely  and  efficiently 
recover  both  heavier  and  faster  aircraft  as  well  as  lighter-weight  unmanned  air 
vehicles  that  may  enter  the  Fleet  in  the  coming  years.  The  NAE  approved  the 
program  for  system  development  and  demonstration  on  February  10,  2005. 

AN/AES-1  Airborne  Laser  Mine  Detection  System.  The  AN/AES-1  Airborne 
Laser  Mine  Detection  System  program  is  under  the  PEO  for  Littoral  and  Mine 
Warfare.  The  system  detects,  classifies,  and  localizes  floating  and  near-surface 
moored  sea  mines.  The  system  is  deployed  from  the  MH-60S  helicopter  and  will 
provide  organic  airborne  mine  defense  to  the  battle  force.  The  NAE  approved  the 
program  for  low-rate  initial  production  on  June  14,  2005. 

AN/AQS-20A  Sonar  Mine  Detecting  Set.  The  AN/AQS-20A  Sonar  Mine 
Detecting  Set  is  under  the  PEO  for  Littoral  and  Mine  Warfare.  It  is  an  upgrade  to 
the  AN/AQS-20  program.  It  can  detect,  localize,  and  classify  bottom,  close- 
tethered,  and  volume  mines.  The  AN/AQS-20A  Sonar  Mine  Detecting  Set 
supports  the  carrier  strike  group  and  the  expeditionary  strike  group.  The  NAE 
approved  the  program  for  low-rate  initial  production  on  May  10,  2005. 

AN/SPY-1D(V)  Radar  Upgrade.  The  AN/SPY-1D(V)  system  reports  through 
the  PEO  for  Integrated  Warfare  Systems.  The  NAE  approved  the  system  for 
low-rate  initial  production  on  January  24,  1997.  The  AN/SPY-1D(V)  system  was 
approved  for  a  second  low-rate  initial  production  on  December  2 1 ,  200 1 .  The 
system  is  an  upgrade  to  the  AN/SPY- ID  to  improve  performance  in  adverse 
natural  environments  and  in  electronic  countermeasures.  The  AN/SPY- 1D(V) 
system  is  a  component  of  the  Aegis  weapon  system. 

AV-8B  Harrier  Open  Systems  Core  Avionics  Requirement.  The  OSCAR 
program  is  under  the  PEO  for  Air  Antisubmarine  Warfare,  Assault,  and  Special 
Mission  Programs.  The  OSCAR  program  is  an  enhancement  to  the  AV-8B 
aircraft  avionics  suite.  It  consists  of  new  avionics  hardware  and  software, 
improved  software  for  a  limited  portion  of  the  existing  avionics  subsystems  and 
Mission  Support  System,  and  integration  of  two  new  weapons,  the  Advanced 
Medium-Range  Air-to-Air  Missile  and  the  Joint  Direct  Attack  Munition.  The 
NAE  approved  the  program  for  full-rate  production  on  August  16,  2004. 

Common  Link  Integration  Processing.  The  CLIP  program  is  under  the  PEO  for 
Command,  Control,  Communications,  Computers,  and  Intelligence  and  Space. 
CLIP  will  enable  several  systems  to  transition  to  Network  Centric  Warfare.  CLIP 
provides  for  an  integration  approach  that  enhances  implementation  of  the  Joint 
Tactical  Radio  System  requirements  through  the  processing  of  the  tactical  data 
link  messages,  provides  gateway  functionality,  and  assists  in  enabling  the 
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Network  Centric  Enterprise  Services  capabilities  as  an  element  of  the  Global 
Information  Grid.  The  NAE  approved  the  program  for  system  development  and 
demonstration  on  June  1,  2005. 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  Upgrade.  The  Mode  5 
program  is  under  the  PEO  for  Tactical  Aircraft  Programs.  Mode  5  is  an  encrypted 
waveform  that  provides  the  warfighter  with  positive,  secure,  and  reliable  line-of- 
sight  identification  of  friendly  aircraft  and  ships.  The  new  capability  to  identify 
friend  or  foe  will  better  support  the  Navy  in  four  operational  environments:  air-to- 
air,  surface-to-air,  air-to-surface,  and  surface-to-surface.  Mode  5  is  an  upgrade  to 
the  Mark  XII  Identification  Friend  or  Foe.  The  NAE  approved  the  program  for 
system  development  and  demonstration  on  August  7,  2003. 

Surface  Electronic  Warfare  Improvement  Program.  SEWIP  is  under  the  PEO 
for  Integrated  Warfare  Systems,  Above  Water  Sensors.  SEWIP  is  an  upgrade  to 
the  existing  AN/SLQ-32  (V)  Electronic  Warfare  System.  SEWIP  includes 
multiple  block  upgrades.  Block  1 A  consists  of  two  subblocks:  the  first  was  for  an 
updated  computer  processor  and  stand-alone  electronic  surveillance  enhancement 
and  the  second  was  for  the  replacement  of  the  existing  display  control  console 
with  an  improved  control  and  display.  The  NAE  approved  the  first  subblock  for 
low-rate  initial  production  on  March  10,  2005,  and  the  second  subblock  for 
low-rate  initial  production  on  December  23,  2005. 
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Appendix  G.  Other  Matters  of  Interest 


Acquisition  Coordination  Teams.  SECNAV  Instruction  5000. 2C, 
“Implementation  and  Operation  of  the  Defense  Acquisition  System  and  the  Joint 
Capabilities  Integration  and  Development  System,”  November  19,  2004,  requires 
that  an  Acquisition  Coordination  Team  be  established  for  each  Navy  ACAT  IC 
and  II  program.  The  13  Navy  programs  reviewed  had  inconsistencies  in  the 
establishment  of  Acquisition  Coordination  Teams.  Specifically,  4  of  the 
13  programs  reviewed  did  not  have  an  established  Acquisition  Coordination 
Team.  One  program,  Harrier  II  AV-8B  OSCAR,  did  not  have  an  Acquisition 
Coordination  Team  or  a  group  with  a  similar  function.  The  program  offices  of  the 
remaining  three  programs  without  an  Acquisition  Coordination  Team  (Advanced 
Arresting  Gear,  APG-79  Active  Electronically  Scanned  Array  Radar  System,  and 
Tactical  Tomahawk)  used  what  they  considered  to  be  a  comparable  functional 
group:  Working  Integrated  Product  Teams  or  Integrating  Integrated  Product 
Teams.  SECNAV  5000.2C  distinguishes  between  Working  Integrated  Product 
Teams,  Integrating  Integrated  Product  Teams,  and  Acquisition  Coordination 
Teams  based  on  the  application  and  functionality  of  each  group.  The  Working 
Integrated  Product  Teams  and  Integrating  Integrated  Product  Teams  are  generally 
formulated  to  address  the  needs  of  various  functional  areas  (for  example,  cost, 
performance,  design,  test,  and  contracting)  or  specific  issues  of  the  program. 
However,  the  SECNAV  guidance  specifically  states  that  an  Acquisition 
Coordination  Team  is  a  team  of  stakeholders  from  the  acquisition  community 
who  represents  the  principal  advisors  to  the  milestone  decision  authority. 

The  programs  with  established  Acquisition  Coordination  Teams  use  the 
Acquisition  Coordination  Team  reviews  during  the  period  leading  up  to  a 
program  decision  meeting.  These  reviews  help  the  program  manager  to  ensure 
that  all  acquisition  documentation  and  other  statutory  and  regulatory  requirements 
have  been  met  prior  to  a  program  decision  meeting  with  the  ASN(RD&A).  The 
existence  of  an  Acquisition  Coordination  Team  supports  the  notion  that  all 
programmatic  issues  should  be  addressed  and  remedied  at  the  lowest  level 
possible,  consistent  with  the  Navy  Integrated  Product  Team  structure,  which  will 
also  result  in  an  effective  procurement  process. 

All  ACAT  IC  and  II  programs  should  have  an  established  Acquisition 
Coordination  Team  that  specifically  addresses  program  readiness  prior  to  a 
program  decision  meeting.  Although  some  of  the  members  of  Working  Integrated 
Product  Teams  and  Integrating  Integrated  Product  Teams  may  overlap  with  key 
members  of  an  Acquisition  Coordination  Team,  Navy  program  officials  should 
establish  a  separate  Acquisition  Coordination  Team  as  required  by  the  SECNAV 
Instruction. 

Common  Link  Integration  Processing.  FAR  15.406-3,  “Documenting  the 
Negotiation,”  states  that  the  contracting  officer  must  document  in  the  contract  file 
the  principal  elements  of  the  negotiated  agreement.  The  Navy  uses  business 
clearance  memorandums  to  summarize  the  negotiated  agreement. 

During  our  site  visit  to  the  CLIP  program  office  on  August  12,  2005,  program 
officials  stated  that  CLIP  did  not  have  a  price  negotiation  memorandum/business 
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clearance  memorandum  for  the  development  contract,  awarded  on  June  9,  2005. 
In  June  2006,  we  followed  up  with  the  contracting  officer  to  request  information 
on  why  CLIP  did  not  have  a  business  clearance  memorandum.  The  contracting 
officer  provided  a  business  clearance  memorandum  dated  June  14,  2006.  The 
business  clearance  memorandum  states  that  a  verbal  clearance  was  provided  by 
Space  and  Naval  Warfare  Systems  Command  on  June  27,  2005,  rather  than  a 
written  clearance  because  the  former  contracting  officer  left  Government  service 
However,  the  former  contracting  officer  did  not  resign  until  August  18,  2005, 
almost  2  months  after  the  contract  was  awarded. 
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Appendix  H.  Management  Comments  on 

Findings  and  Audit  Response 


The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  provided 
comments  on  behalf  of  the  ASN(RD&A). 

Management  Comments  on  AN/APG-79  Active  Electronically  Scanned 
Array  Radar  System  Acquisition  Strategy.  The  Deputy  Assistant  Secretary  of 
the  Navy  (Management  and  Budget)  stated  that  the  assertion  that  the  approval  of 
the  AN/APG-79  Active  Electronically  Scanned  Array  Radar  System’s  acquisition 
strategy  was  caused  by  an  oversight  in  the  NAE  approval  chain  was  misleading. 
According  to  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and 
Budget),  the  ASN(RD&A)  directed  the  program  office  to  make  a  revision  to  the 
program  involving  a  fourth  LRIP  decision  at  the  Milestone  C  decision  review. 

The  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that 
this  decision  required  the  program  office  to  revise  the  acquisition  strategy, 
causing  a  delay  in  the  fonnal  approval  of  the  acquisition  strategy  (page  86). 

Audit  Response.  We  revised  the  report  (page  9)  to  reflect  that  the  delay  was 
caused  by  the  ASN(RD&A)  decision  made  during  the  Milestone  C  decision 
review. 

Management  Comments  on  Mode  5  Acquisition  Strategy.  The  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that  the 
ASN(RD&A)  signed  the  acquisition  strategy  6  months  after  the  acquisition 
decision  memorandum  for  Milestone  B  because  the  ASN(RD&A)  directed  the 
Mode  5  program  to  make  changes  to  the  acquisition  strategy  during  the 
Milestone  B  decision  review.  According  to  the  Deputy  Assistant  Secretary  of  the 
Navy  (Management  and  Budget),  the  required  changes  were  made  and  the 
document  was  restaffed;  however,  the  document  did  not  need  to  be  resigned  due 
to  the  ASN(RD&A)  concurrence  with  the  changes.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  also  stated  that  upon 
concurrence  with  the  other  signature  authorities,  the  document  was  then  sent  to 
ASN(RD&A)  for  final  signature  (pages  86-87). 

Audit  Response.  DoD  Instruction  5000.2  requires  an  approved  acquisition 
strategy  at  the  Milestone  B  review.  It  does  not  state  that  documents  changed 
because  of  a  milestone  decision  are  not  required  to  be  resigned.  If  the  acquisition 
strategy  was  not  resigned,  the  ASN(RD&A)  would  have  no  way  of  determining  if 
appropriate  changes  were  made. 

Management  Comments  on  Common  Link  Integration  Processing  AoA.  The 

Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  agreed  that 
SECNAV  Instruction  5000. 2C  requires  the  AoA  for  the  Common  Link 
Integration  Processing  program  to  be  approved  by  the  NAE,  the  Chief  of  Navy 
Operations,  or  the  Commandant  of  the  Marine  Corps.  However,  the  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  cited  the  SECNAV 
Instruction  as  stating  the  PEO  has  overall  responsibility  of  the  AoA.  According 
to  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget),  the 
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program  office  interpreted  that  level  of  responsibility  as  providing  adequate 
authority  for  the  PEO  to  approve  the  AoA  (page  87). 

Audit  Response.  Although  SECNAV  Instruction  5000. 2C  gives  overall 
responsibility  of  the  AoA  to  the  PEO,  it  does  not  give  authority  to  the  PEO  to 
approve  the  AoA.  According  to  the  Instruction,  the  Component  Acquisition 
Executive,  the  milestone  decision  authority,  the  Chief  of  Naval  Operations,  or  the 
Commandant  of  the  Marine  Corps  has  the  authority  to  approve  the  AoA  unless 
the  milestone  decision  authority  designates  that  authority  to  another  official,  such 
as  the  PEO.  Based  on  our  review  of  the  Common  Link  Integration  Processing 
program,  we  found  no  evidence  that  the  milestone  decision  authority  delegated 
the  AoA  approval  to  the  PEO. 

Management  Comments  on  AN/SPY-1D(V)  Radar  Upgrade.  The  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  commented  that  our 
statement,  “the  system  is  an  upgrade  to  the  AN/SPY- 1D(V)  to  improve 
performance  in  adverse  environments  with  natural  and  electronic 
countermeasures”  is  incorrect.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  stated  that  the  statement  should  read  “the  system  is  an 
upgrade  to  the  AN/SPY- ID  to  improve  perfonnance  in  adverse  natural 
environments  and  in  electronic  countermeasures”  (page  87). 

Audit  Response.  We  revised  the  statement  in  the  audit  report  (page  60). 

Management  Comments  on  SEWIP  Human  Systems  Integration  Plan.  The 

Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  an 
independent  Human  Systems  Integration  plan,  which  encompasses  the  entire 
SEWIP  program,  was  completed  May  12,  2003,  and  approved  January  31,  2006, 
in  accordance  with  the  DoD  Instruction  5000.2.  The  Deputy  Assistant  Secretary 
of  the  Navy  (Management  and  Budget)  also  stated  that  the  milestone  decision 
authority  approved  the  acquisition  documentation  for  the  SEWIP  program  on 
August  13,  2002,  per  the  “Acquisition  Decision  Memorandum  for  the  Surface 
Electronic  Warfare  Improvement  Program”  (page  88). 

Audit  Response.  We  agree  that  a  Humans  Systems  Integration  Plan  was 
completed  for  the  SEWIP  program.  However,  it  was  approved  after  both 
increments  of  Block  1 A  were  approved  for  Milestone  C  and  was  not  included  as 
part  of  the  acquisition  strategy.  DoD  Instruction  5000.2  requires  the  program 
manager  to  have  a  comprehensive  plan  for  human  systems  integration  early  in  the 
acquisition  process  to  optimize  total  system  performance,  minimize  total 
ownership  costs,  and  ensure  that  the  system  is  built  to  accommodate  the 
characteristics  of  the  user  population  that  will  operate,  maintain,  and  support  the 
system.  Because  the  plan  was  not  approved  until  after  both  increments  of  Block 
1 A  were  approved  for  Milestone  C,  the  plan  was  not  authorized  early  in  the 
acquisition  process.  In  addition,  during  our  audit,  SEWIP  program  officials  could 
not  provide  the  tailoring  agreement  referenced  in  the  August  13,  2002,  acquisition 
decision  memorandum.  Therefore,  we  could  not  validate  that  one  was  developed 
and  approved. 

Management  Comments  on  AN/AES-1  Airborne  Laser  Mine  Detection 
System  Operational  Assessment  Testing.  The  Deputy  Assistant  Secretary  of 
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the  Navy  (Management  and  Budget)  stated  that  the  program  office  agrees  that 
operational  assessment  testing  had  not  been  conducted  on  the  AN/AES-1 
Airborne  Laser  Mine  Detection  System  prior  to  the  LRIP  decision.  He  stated 
further  that  program  office  did  conduct  the  testing  that  was  required  by  the 
acquisition  decision  memorandum  for  an  LRIP  decision.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  stated  that  a  developmental 
test-assist  was  conducted  concurrently  with  Commander,  Operational  Test  and 
Evaluation  Force  and  the  results  documented  in  a  Letter  of  Observation,  as 
required  by  the  acquisition  decision  memorandum  (pages  88-89). 

Audit  Response.  DoD  Instruction  5000.2  states  that  entrance  into  Milestone  C 
depends,  in  part,  on  acceptable  performance  in  development  test  and  evaluation; 
operational  assessment  testing;  and  acceptable  operational  supportability.  A 
developmental  test-assist  is  not  a  formal  phase  of  operational  testing  and  does  not 
resolve  critical  operational  issues,  nor  does  it  reach  conclusions  regarding 
operational  effectiveness  and  suitability  or  recommendations  regarding  fleet 
introduction.  Therefore,  operational  assessment  testing  should  have  been 
preformed. 

In  addition,  the  Letter  of  Observation  states  that  performance  upgrades  and 
modifications  are  planned  for  LRIP  and  must  be  thoroughly  tested  to  verify 
performance;  there  is  no  post-mission  analysis  training  for  fleet  personnel;  and 
operational  requirements  document  thresholds  have  been  waived  for  the  LRIP 
configuration,  but  not  the  production  units.  The  Commander,  Operational  Test 
and  Evaluation  Force  stated  that  performance  has  slowly  improved,  and  in  some 
cases,  the  waived  thresholds  have  been  met.  However,  the  extremely  limited 
number  of  successful  flights  makes  it  difficult  to  detennine  the  performance  or 
future  potential  of  the  system.  The  system  must  show  progress  toward,  and 
potential  to  meet,  full  operational  requirements  document  thresholds,  as  well  as 
mission  repeatability,  before  it  will  be  ready  to  proceed  to  operational  evaluation. 
Programs  should  not  be  approved  to  enter  into  the  production  and  deployment 
phase  of  the  acquisition  process  until  the  system  has  demonstrated,  during 
operational  testing,  that  it  is  potentially  operationally  effective  and  suitable  and 
will  meet  the  users  needs. 

Management  Comments  on  Command,  Control,  Communications, 
Computers,  and  Intelligence  Support  Plan  Waiver.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  stated  the  ORD  Consolidation 
Letter  for  OSCAR  dated  January  29,  2002,  along  with  the  series  of  documents 
which  it  referenced  and  validated,  properly  established  the  requirements  for  the 
OSCAR  program.  Additionally,  he  stated  that  the  requirements  did  not  include  an 
interoperability  key  performance  parameter.  The  Deputy  Assistant  Secretary  of 
the  Navy  (Management  and  Budget)  stated  that  the  fact  that  the  command, 
control,  communications,  computers,  and  intelligence  support  plan  waiver  was 
signed  by  Deputy  Assistant  Secretary  of  the  Navy  (Command,  Control, 
Communications,  Computers,  and  Intelligence  and  Space)  instead  of  by  the 
ASN(RD&A)  does  not  alter  the  fundamental  validity  of  the  waiver  request  or 
imply  that  the  program  should  have  developed  a  command,  control, 
communications,  computers,  and  intelligence  support  plan  (pages  89-90). 
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Audit  Response.  Although  program  officials  for  OSCAR  obtained  a  command, 
control,  communications,  computers,  and  intelligence  support  plan  waiver  from 
Deputy  Assistant  Secretary  of  the  Navy  (Command,  Control,  Communications, 
Computers,  and  Intelligence  and  Space),  the  waiver  did  not  have  the  correct 
approval  authority  and  the  waiver  was  based  on  inappropriate  ORDs.  DoD 
Regulation  5000.2-R,  “Mandatory  Procedures  for  Major  Defense  Acquisition 
Programs  and  Major  Automated  Information  System  Acquisition  Programs,”  June 
10,  2001,  states  that  the  NAE,  with  the  advice  from  the  appropriate  chief 
information  officer,  may  waive  command,  control,  communications,  computers, 
and  intelligence  support  plan  preparation  if  the  requirements  authority  has 
previously  waived  the  requirement  for  an  interoperability  key  performance 
parameter  in  the  ORD.  The  waiver  was  not  approved  by  the  NAE  as  required  by 
DoD  Regulation  5000.2-R.  As  a  result,  the  approval  authority  for  the  waiver  was 
invalid. 

In  addition,  the  approval  of  the  waiver  was  based  on  inappropriate  ORDs.  In  a 
Department  of  the  Navy  memorandum,  March  26,  2003,  the  Office  for  Chief  of 
Naval  Operations  supported  waiving  the  requirement  for  an  OSCAR  command, 
control,  communications,  computers,  and  intelligence  support  plan  for  the 
following  reasons. 

•  The  AV-8B  Radar  and  Night  Attack  ORDs,  which  govern  the  OSCAR 
program,  do  not  include  an  interoperability  key  perfonnance  parameter. 

•  The  AV-8B  Radar  and  Night  Attack  ORDs  do  not  include  an  external 
information  exchange  requirements. 

•  Of  the  two  major  components  that  make  up  the  OSCAR  program,  the 
mission  computer  already  has  a  command,  control,  communication, 
computer  and  intelligence  support  plan  waiver,  and  the  weapons  computer 
has  no  connection  to  the  communications  and  information  infrastructure. 

However,  in  our  review  of  the  Test  and  Evaluation  Master  Plan  we  found  that  the 
OSCAR  program  is  testing  requirements  found  in  additional  ORDs  that  are  not 
listed  in  the  memorandum.  Those  additional  requirements  are: 

•  Joint  Direct  Attack  Munition  ORD, 

•  ARC-210  ORD,  and 

•  AV-8B  Targeting  Pod  Requirements  Letter. 

Therefore,  because  the  waiver  was  not  properly  approved  by  the  NAE  and  the 
memorandum  asking  for  approval  does  not  contain  all  the  requirements  for  the 
OSCAR  system,  we  feel  the  waiver  is  not  valid.  In  addition  to  this  audit,  DoD  IG 
Report  No.  D-2004-109,  “Implementation  of  the  DoD  Management  Control 
Program  for  Navy  Acquisition  Category  II  and  III  Programs,”  August  17,  2004, 
found  that  the  program  sponsor  did  not  develop  an  ORD  for  the  OSCAR  program 
as  required  by  DoD  Instruction  5000.2. 


66 


Management  Comments  on  OSCAR-Specific  ORD.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  stated  that  there  are  six 
requirements  documents  that  collectively  define  the  operational  performance 
requirement  for  the  OSCAR  upgrade.  In  addition,  the  Deputy  Assistant  Secretary 
of  the  Navy  (Management  and  Budget)  stated  that  the  ORD  Consolidation  Letter 
for  OSCAR,  dated  January  29,  2002,  specifically  defined  the  performance 
parameters  of  the  OSCAR  upgrade  based  on  the  performance  requirements 
defined  in  the  six  ORDs  (page  90). 

Audit  Response.  DoD  5000-2R  requires  that  users  develop  ORDs  to  illustrate 
program  thresholds  and  objectives.  These  program  thresholds  and  objectives  are 
measures  of  effectiveness  or  performance  and  minimum  acceptable  requirements 
for  the  program.  We  found  that  the  ORD  Consolidation  Letter  only  references 
two  of  the  ORDs  that  were  mentioned  in  the  comments  from  the  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget).  Because  not  all  of  the  ORDs 
were  referenced  in  the  ORD  Consolidation  Letter,  there  is  no  evidence  that  the 
ORD  Consolidation  Letter  fully  defines  the  operational  perfonnance  requirement 
for  the  OSCAR  upgrade.  In  addition,  a  previous  report,  DoD  IG  Report  No.  D- 
2004-109,  “Implementation  of  the  DoD  Management  Control  Program  for  Navy 
Acquisition  Category  II  and  III  Programs,”  August  17,  2004,  states  that  an  ORD 
was  not  prepared  or  approved  for  the  OSCAR  program.  Based  on  the  previous 
audit  and  our  findings  in  this  report,  we  believe  that  the  OSCAR  program  should 
have  developed  an  OSCAR-specific  ORD. 

Management  Comments  on  the  MH-60R  Multi-Mission  Helicopter  Test  and 
Evaluation  Master  Plan.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  stated  that  the  MH-60R  program  had  an  approved  Test 
and  Evaluation  Master  Plan,  Revision  B,  that  indicated  the  operational  testing 
required  for  the  Milestone  C.  He  further  stated  that  the  Commander,  Operational 
Test  and  Evaluation  Force  provided  the  necessary  approval  to  support  the  aircraft 
for  full-rate  production  as  tested.  In  addition,  the  Test  and  Evaluation  Master 
Plan  was  updated  and  submitted  for  approval  prior  to  Milestone  C.  Revision  C  of 
the  Test  and  Evaluation  Master  Plan  was  approved  through  the  Director,  Test  and 
Evaluation  and  by  the  milestone  decision  authority  and  briefed  to  Director, 
Operational  Test  and  Evaluation  staff  prior  to  the  milestone.  The  Deputy 
Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that  the 
Director,  Operational  Test  and  Evaluation  staff  agreed  with  the  scope  of  follow- 
on  testing  and  was  in  attendance  at  the  milestone  review  (page  91). 

Audit  Response.  SECNAV  Instruction  5000.2C  requires  the  program  manager 
to  submit  an  approved  test  and  evaluation  master  plan  as  part  of  the  full-rate 
production  decision  review.  It  must  be  approved  by  Director,  Operational  Test 
and  Evaluation  for  AC  AT  IC  programs  such  as  MH-60R.  The  Test  and 
Evaluation  Master  Plan,  Revision  C,  was  not  approved  by  Director,  Operational 
Test  and  Evaluation  for  ACAT  IC  programs  prior  to  the  full-rate  production 
decision  on  March  31,  2006;  therefore,  the  program  did  not  meet  the  SECNAV 
Instruction  5000. 2C  requirement. 

Management  Comments  on  the  MH-60R  Multi-Mission  Helicopter 
Operational  Assessment  Testing.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Management  and  Budget)  stated  that  even  though  the  operational  assessment  was 
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stopped  by  the  program  office,  the  Commander,  Operational  Test  and  Evaluation 
Force  reported  the  findings  of  the  operational  assessment.  He  also  stated  that  the 
program  office  addressed  each  of  the  findings  and  the  plans  to  correct  them.  The 
Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget)  stated  that  the 
NAE  approved  LRIP  II  and  the  resultant  acquisition  decision  memorandum 
references  the  findings  from  the  operational  assessment.  The  acquisition  decision 
memorandum  specifically  identifies  near-tenn  criteria  for  the  program  to  meet  in 
order  to  proceed  to  the  LRIP  III.  In  June  2003,  the  Commander,  Operational  Test 
and  Evaluation  Force  changed  the  policy  for  reporting  on  operational  assessments 
to  identify  the  risk  areas  of  the  program  (pages  91-92). 

Audit  Response.  DoD  Instruction  5000.2  states  that  entrance  into  Milestone  C 
depends  on  acceptable  performance  in  development,  test  and  evaluation; 
operational  assessment  testing;  acceptable  operational  supportability;  and  mature 
software  capability.  In  March  2002,  the  ASN(RD&A)  approved  revised  exit 
criteria  for  the  MH-60R  LRIP  II  to  include  completion  of  Operational  Testing  II- 
A  with  a  finding  of  potentially  operationally  effective  and  potentially 
operationally  suitable.  An  operational  assessment  was  conducted  from  June 
through  September  2003  to  determine  the  potential  operational  effectiveness  and 
potential  operational  suitability  of  the  MH-60R.  The  operational  assessment  was 
tenninated  by  the  program  office  prior  to  completion  of  the  testing  due  to  poor 
performance  of  the  radar,  electronic  support  measures,  and  acoustic  subsystems; 
software  not  mature  enough  for  operational  testing;  and  post-mission  analysis 
training  did  not  ensure  proficiency.  The  NAE  approved  the  program  for  LRIP  II 
on  December  15,  2003,  even  though  an  operational  assessment  was  not  completed 
as  required  by  the  acquisition  decision  memorandum  dated  March  14,  2002.  In 
addition,  no  justification  was  included  in  the  acquisition  decision  memorandum 
for  proceeding  with  LRIP  without  the  required  operational  assessment.  Also,  a 
second  operational  assessment  was  conducted  from  October  to  December  2004. 
The  operational  assessment  report,  March  18,  2005,  stated  that  the  system  did  not 
meet  all  threshold  values  for  operational  effectiveness  and  operational  suitability 
and  included  a  listing  of  recommendations  that,  if  not  addressed,  could  result  in 
the  failure  of  operational  evaluation  testing.  However,  the  NAE  approved 
LRIP  III,  in  an  acquisition  decision  memorandum  dated  April  5,  2005. 

Programs  should  not  be  approved  to  enter  into  the  production  and  deployment 
phase  of  the  acquisition  process  until  the  system  has  demonstrated  during 
operational  testing  that  it  is  potentially  operationally  effective  and  suitable  and 
will  meet  the  users  needs.  Doing  so  could  result  in  systems  being  produced  that 
will  not  meet  the  users’  needs  and  require  additional  funds  to  retrofit  the  systems 
that  have  already  been  produced.  In  addition,  approval  of  milestone  decisions 
without  meeting  previously  approved  exit  criteria  should  be  documented  in  the 
acquisition  decision  memorandum. 

Management  Comments  on  Tactical  Tomahawk.  The  Deputy  Assistant 
Secretary  of  the  Navy  (Management  and  Budget)  stated  that  the  report  incorrectly 
states  that  the  Tactical  Tomahawk  program  does  not  have  an  acquisition  strategy. 
He  also  stated  that  the  ASN(RD&A)  determined  that  because  Tactical  Tomahawk 
is  a  product  improvement  building  upon  the  Block  III  program,  an  AoA  was  not 
required  for  Milestone  III.  Program  conditions  had  not  changed  significantly.  As 
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a  result,  the  Deputy  Assistant  Secretary  of  the  Navy  (Management  and  Budget) 
does  not  feel  that  the  document  was  applicable  to  the  program  (pages  92-93). 

Audit  Response.  During  our  audit,  we  were  not  provided  a  signature  page 
verifying  the  ASN(RD&A)  approval  of  the  full-rate  production  Tactical 
Tomahawk  acquisition  strategy.  After  we  issued  the  draft  report,  ASN(RD&A) 
officials  provided  us  with  the  full-rate  production  acquisition  strategy,  signed  on 
September  12,  2003,  by  the  ASN(RD&A).  We  updated  the  report  to  reflect  that 
Tactical  Tomahawk  has  an  approved  acquisition  strategy. (pages  6  &  8) 

We  do  not  agree  that  an  AoA  is  not  applicable  for  Tactical  Tomahawk.  An  AoA 
is  the  evaluation  of  the  operational  effectiveness,  operational  suitability,  and 
estimated  costs  of  alternative  systems  to  meet  a  mission  capability.  The  analysis 
assesses  the  advantages  and  disadvantages  of  alternative  systems  being 
considered  to  satisfy  a  validated  need,  including  the  sensitivity  of  each  alternative 
to  possible  changes  in  key  assumptions  and  variables.  Although  Tactical 
Tomahawk  is  an  upgrade  to  the  previous  Tactical  Tomahawk  blocks,  program 
officials  should  have  developed  an  AoA  at  the  conception  of  the  Tactical 
Tomahawk  program. 
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Appendix  I.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Director,  Acquisition  Resources  and  Analysis 
Director,  Defense  Procurement  and  Acquisition  Policy 
Under  Secretary  of  Defense  (Comp troller)/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Director,  Program  Analysis  and  Evaluation 


Joint  Staff 

Director,  Joint  Staff 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs) 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 

Deputy  Assistant  Secretary  of  the  Navy 
Air  Programs 
Acquisition  Management 

Command,  Control,  Communications,  Computers,  and  Intelligence  and  Space 
Programs 

Integrated  Warfare  Systems 
Littoral  and  Mine  Warfare  Programs 
Management  and  Budget 

Director,  Air  1 .0  Program  Management  for  Advanced  Arresting  Gear 
Program  Manager,  Advanced  Arresting  Gear 
Program  Executive  Office,  Air  Antisubmarine  Warfare,  Assault,  and  Special  Mission 
Programs 

Program  Manager,  AV-8B  Harrier  II  Open  Systems  Core  Avionics  Requirements 
Program  Manager,  MH-60R  Multi-Mission  Helicopter 
Program  Executive  Officer,  Command,  Control,  Communications,  Computers,  and 
Intelligence  and  Space 

Program  Manager,  Navy  Advanced  Extremely  High  Frequency  Multiband  Terminal 
Program  Manager,  Common  Link  Integration  Processing 
Program  Executive  Officer,  Integrated  Warfare  Systems 
Program  Manager,  AN/SPY- 1D(V)  Radar  Upgrade 
Program  Manager,  Surface  Electronic  Warfare  Improvement  Program 
Program  Executive  Officer,  Littoral  and  Mine  Warfare 
Program  Manager,  Advanced  Deployable  System 
Program  Manager,  AN/AES-1  Airborne  Laser  Mine  Detection  System 
Program  Manager,  AN/AQS-20A  Sonar  Mine  Detecting  Set 
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Department  of  the  Navy  (cont’d) 

Program  Executive  Officer,  Strike  Weapons  and  Unmanned  Aviation 
Program  Manager,  Tactical  Tomahawk 
Program  Executive  Officer,  Tactical  Aircraft  Programs 

Program  Manager,  APG-79  Active  Electronically  Scanned  Array  Radar  System 
Program  Manager,  Mark  XIIA  Mode  5  Identification  Friend  or  Foe  Upgrade 
Naval  Inspector  General 
Auditor  General,  Department  of  the  Navy 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 

Senate  Committee  on  Homeland  Security  and  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 

House  Committee  on  Anned  Services 

House  Committee  on  Oversight  and  Government  Refonn 

House  Subcommittee  on  National  Security  and  Foreign  Affairs,  Committee  on  Oversight 
and  Government  Refonn 
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Department  of  the  Navy  Comments 


DEPARTMENT  OF  THE  NAVY 

OFFICE  OF  THE  ASSISTANT  SECRETARY 
RESEARCH,  DEVELOPMENT  AND  ACQUISITION 
1000  NAVY  PENTAGON 
WASHINGTON  DC  20350-1000 

DEC  7  2006 

MEMORANDUM  FOR  OFFICE  OF  THE  INSPECTOR  GENERAL  OF  THE 
DEPARTMENT  OF  DEFENSE 

Subj:  REPORT  ON  THE  NAVY  ACQUISITION  EXECUTIVE’S 

MANAGEMENT  OVERSIGHT  AND  PROCUREMENT  AUTHORITY 
FOR  ACQUISITION  CATEGORY  I  AND  n  PROGRAMS 

Ref:  (a)  DODIG  Draft  Report  No.  N2005-D00(K)AS-0230.000  of  Oct  3 1 ,  2006 

Enel:  (1)  Summary  of  Recommendations  and  Actions  take  for  DODIG  No. 
N2005-D0O00AS-0230.000  Draft  Report 
(2)  Summary  of  Comments  to  Various  Findings  in  subject  Draft  Report 

In  response  to  reference  (a),  enclosures  ( I )  and  (2)  are  forwarded  listing  the 
comments,  recommendations  and  status  of  action  taken.  Questions  concerning 
this  letter  should  be  directed  to  Ms  Katherine  Cewe,  OASN(RD&A)(M&B)  who 
can  be  reached  at  (703)  614-0144,  or  e-mail  Katie.Cewe@navv.mil. 


John  S.  Thackrah 

Deputy  Assistant  Secretary  of  the  Navy 
(Management  &  Budget) 


Copy  to: 

OASN(RD&A)(CONG) 

NAVIG 

PEO(A) 

PEO(C41  and  Space) 
PEO(IWS) 

PEO(LMW) 

PEO(T) 

PEO(W) 

AIR  1.0 
AIR  OOG4 
SEA  00N3C 
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DEPARTMENT  OF  THE  NAVY  RESPONSE  TO  RECOMMENDATIONS 
AND  ACTION  TAKEN  REGARDING  DODIG  DRAFT  REPORT  ON  NAVY 
ACQUISITION  EXECUTIVE  S  MANAGEMENT  OVERSIGHT  AND 
PROCUREMENT  AUTHORITY  FOR  ACQUISTIION  CATEGORY  I  AND  II 
PROGRAMS  (PROJECT  NO.  D2005-D000AS-023.000) 

Recommendation  A.  1  a: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  comply  with  Under  Secretary  of  Defense  Acquisition, 

Technology,  and  Logisitics  Memorandum.  “Policy  for  Systems  Engineermg  in 
DoD."  February  20.  2004:  DoD  Instruction  5000.2:  and  Secretary  of  the  Navy 
Instruction  5000. 2C  when  reviewing  and  approving  systems  engineering  plans  as 
part  of  a  program's  milestone  review.” 

DON  Comments: 

Concur.  In  June  2005  the  Assistant  Secretary  of  the  Navy  for  Research. 
Development  and  Acquisition  (ASN  (RD&A))  promulgated  DoN  policy  and 
process  for  SEP  review  and  approval  (ASN(RD&A)  Memorandum.  "Policy  for 
DoN  Systems  Engineering  Plan  (SEP)  Review  and  Approval,"  June  6,  2005)  to 
implement  the  requirements  of  the  USD  (AT&L)  memorandum.  In  April  2006, 
the  ASN(RD&A)  consolidated  responsibility  for  SEP  development  and  approval 
with  the  ASN  (RD&A)  Clnef  Systems  Engineer  (ASN(  RD&A)(CHENG))  (ASN 
(RD&A)  Memorandum.  "Revised  Policy  for  DoN  Systems  Engineering  Plan  (SEP) 
Review  and  Approval,"  April  27,  2006).  In  accordance  with  NAE  direction  of  27 
April  2006,  ASN(RD&A)(CHENG)  has  been  delegated  the  responsibility  for 
approval  of  all  ACAT  IC.  IAC  and  II  System  Engineering  Plans.  Additionally. 

SEP  Component  approval  authority  was  also  delegated  for  ACAT  ID  and  IAM 
programs  to  ASN(RD&A)(C'HENG).  The  responsibilities  of  the  Program 
Executive  Officers  (PEOs)  and  Systems  Commanders  (SYSCOMs)  to  ensure 
timely  development  and  implementation  of  quality  SEPs  was  emphasized  with 
ASN(RD&A)(CHENG)  as  the  central  office  of  primary  responsibility  for  SEP 
approval.  In  addition,  the  SECNAVINST  5000. 2D  has  been  updated  accordmgly. 


Recommendation  A.  1  b: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  review,  sign,  date  and  maintain  decision  documents  in  the 
program  files  as  a  result  of  milestone  reviews  in  the  acquisition  decision 
memorandum,  waiver,  or  tailored  documentation  agreement.” 
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DON  Comments: 

Concur  ASN(RD&A)  will  formally  document  decisions  regarding  tailoring  or 
waiver  of  documentation  in  future  acquisition  decision  memoranda,  a  tailoring 
agreements  or  a  waiver  approval.  Also,  these  documents  will  be  maintained  in  the 
program  file  for  future  reference. 

Recommendation  A.  1  c: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research.  Development, 
and  Acquisition)  comply  with  the  Defense  Acquisition  Guidebook 
recommendations  to  update  and  approve  acquisition  program  baselines  for 
programs  between  milestone  decision  reviews  within  90  clays.” 

DON  Comments: 

Partially  Concur,  where  feasible.  The  DON  Acquisition  and  Capabilities 
Guidebook,  Section  1.3.1  states  that  within  90  days  of  the  program  deviation,  the 
PM  is  to  1)  ensure  the  program  is  back  within  APB  thresholds,  or  2)  submit  a  new 
APB,  changing  only  the  breached  parameters  and  those  parameters  directly 
affected  by  the  breached  parameter,  or,  3)  provide  a  date  by  which  the  new  APB 
will  be  submitted  or  by  which  the  program  will  be  back  within  original  APB 
thresholds,  and  4)  keep  the  C'NO  CMC  (DC.P&R.  and  CD)  informed  with  regard 
to  program  deviations  and  baseline  recovery  actions.  The  Defense  Acquisition 
Guidebook  states  that  “Within  90  days  of  the  occurrence  of  the  program  deviation, 
one  of  the  following  should  have  occurred:  the  program  is  back  within  APB 
parameters;  a  new  APB  (changing  only  those  parameters  that  wrere  breached)  has 
been  approved;  or  an  OIPT-level  program  review'  has  been  conducted  to  review 
the  program  manager’s  proposed  baseline  revisions  and  make  recommendations  to 
the  Milestone  Decision  Authority.  Approval  by  the  MDA  of  the  revised  APB  is 
desired  to  be  obtained  within  90  days  after  the  reporting  of  the  deviation,  however, 
it  is  not  always  feasible.  Instead  w'hen  necessary.  Program  Review's  are  scheduled 
to  decide  on  the  future  course  of  the  program  or  alternatives  and  direction  may  be 
given  as  to  the  contents  of  the  program  of  record.  Given  the  number  of  AC  AT 
programs,  the  volatility  of  these  efforts,  the  normal  time  required  to  develop  an 
Independent  Cost  Estimate,  and  all  other  administrative  issues  associated  with 
APB  revisions,  approval  by  the  MDA  is  not  ahvays  feasible  withm  a  90  day 
timeframe.  As  such,  the  DON  Acquisition  and  Capabilities  Guidebook.  Section 
1 .4.4  places  no  such  requirement  on  the  MDA.  The  Defense  Acquisition 
Guidebook  is  considered  guidance  for  AC  AT  I  level  programs,  but  not  regulatory 
or  required.  Department  of  the  Navy  desu  es  and  strives  to  make  every  effort  to 
achieve  approval  of  revised  APBs  as  expediently  as  possible. 
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Recommendation  A.  1  d: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  track  deviation  reports  in  the  Navy  Dashboard  database  to 
include  the  date,  cause,  effect,  and  mitigation.” 

DON  Comments: 

Concur  ASN(RD&A)  will  make  the  necessary  changes  to  the  Navy  Dashboard 
database  and  start  tracking  deviation  reports  and  baseline  breaches  (to  include  date, 
cause,  effect  and  mitigation). 

Recommendation  A.  1  ,e: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  approve  or  disapprove  updated  acquisition  program  baselines 
within  90  day  of  receiving  them  from  the  program  manager.” 

DON  Comments: 

Partially  Concur,  where  feasible.  The  DON  Acquisition  and  Capabilities 
Guidebook.  Section  1.3.1  states  that  within  90  days  of  the  program  deviation,  the 
PM  is  to  1)  ensure  the  program  is  back  within  APB  thresholds,  or  2)  submit  a  new 
APB.  changing  only  the  breached  parameters  and  those  parameters  directly 
affected  by  the  breached  parameter,  or,  3)  provide  a  date  by  which  the  new  APB 
will  be  submitted  or  by  which  the  program  will  be  back  within  original  APB 
thresholds,  and  4)  keep  the  CNO  CMC  (DCP&R.  and  C'D)  informed  with  regard 
to  program  deviations  and  baseline  recovery  actions.  The  Defense  Acquisition 
Guidebook  states  that  “Within  90  days  of  the  occurrence  of  the  program  deviation, 
one  of  the  following  should  have  occurred:  the  program  is  back  within  APB 
parameters;  a  new  APB  (changing  only  those  parameters  that  were  breached)  has 
been  approved;  or  an  OIPT-level  program  review  has  been  conducted  to  review 
the  program  manager’s  proposed  baseline  revisions  and  make  recommendations  to 
the  Milestone  Decision  Authority.  Approval  by  the  MDA  of  the  revised  APB  is 
desired  to  be  obtained  within  90  days  after  the  reporting  of  the  deviation,  however, 
it  is  not  always  feasible.  Instead  when  necessary,  Program  Reviews  are  scheduled 
to  decide  on  the  future  course  of  the  program  or  alternatives  and  direction  may  be 
given  as  to  the  contents  of  the  program  of  record.  Given  the  number  of  AC'AT 
programs,  the  volatility  of  these  efforts,  the  normal  tune  required  to  develop  an 
Independent  Cost  Estimate,  and  all  other  administrative  issues  associated  with 
APB  revisions,  approval  by  the  MDA  is  not  always  feasible  within  a  90  day 
timeframe.  As  such,  the  DON  Acquisition  and  Capabilities  Guidebook,  Section 
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1 .4.4  places  no  such  requirement  on  the  MDA.  The  Defense  Acquisition 
Guidebook  is  considered  guidance  for  AC  AT  I  level  programs,  but  not  regulatory 
or  required.  Department  of  the  Navy  desires  and  strives  to  make  every  effort  to 
achieve  approval  of  revised  APBs  as  expediently  as  possible. 

Recommendation  A. 2: 

“We  recommend  that  program  managers  maintain  approved  documentation  for  the 
life  of  the  acquisition  program.” 

DON  Comments: 

Concur  The  Program  Offices  are  expected  to  maintain  appropriate  repositories  of 
program  documentation  for  the  life  cycle  of  the  acquisition  program.  ASN(RD&A) 
will  issue  additional  guidance  to  the  various  Program  Offices  to  reinforce  tins 
requirement. 

Recommendation  A.3 : 

“We  recommend  that  the  AV-SB  Harrier  II  Open  System  Core  Avionics 
Requirements  program  manager  develop  and  prepare  a  tailored  Information 
Support  Plan  as  required  by  DoD  Instruction  5000.2.” 

DON  Comments: 

Do  not  concur.  At  the  time  of  OSCAR  s  development,  there  were  no  specific 
interoperability  requirements  for  the  OSCAR  system  beyond  the  legacy  system  it 
replaced,  no  external  Information  Exchange  Requirements,  and  no  interoperability 
KPPs.  For  these  reasons,  a  C4ISP  waiver  was  granted  on  25  Aug  03.  by  Deputy 
Assistant  Secretary  of  the  Navy  for  C4I  and  Space.  Dr.  D.G.  Uhler.  The  NAE  was 
aware  of  the  waiver  and  agreed  with  it.  Consequently,  the  Navy  believes  that  the 
original  waiver  was  properly  processed  and  approved  The  rationale  for  the 
original  waiver  request  was  valid  and  remains  valid.  Therefore,  there  is  no  more 
of  a  requirement  for  an  ISP  today  than  there  was  at  the  time  that  the  waiver 
request  was  submitted.  The  OSCAR  program  is  currently  58%  delivered  to  the 
Fleet.  Retroactively  completing  an  ISP  at  this  point  in  the  program's  life  cycle 
would  be  an  inefficient  use  of  available  resources  with  limited  or  no  benefit  to  the 
program  or  the  Fleet. 


Revised 


77 


Final  Report 
Reference 


Revised 


Recommendation  A. 4: 

“We  recommend  that  the  Surface  Electronic  Warfare  Improvement  Program  and 
the  AN  AES- 1  Airborne  Laser  Mine  Detection  System  program  managers  submit 
their  systems  engineering  plans  for  Navy  Acquisition  Executive  approval  as 
required  by  Under  Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics) 
Memorandum.  “Policy  for  Systems  Engineering  in  DoD,”  February  20.  2004.” 

DON  Comments: 

Concur  The  SEWIP  System  Engineering  Plan  was  recently  developed,  submitted 
and  approved  on  22  Oct  2006.  In  addition,  and  in  order  to  maintain  the  relevancy 
and  effectiveness  of  the  SEWIP  SEP,  ASN  (RD&A)  (CHENG)  has  tasked  the 
SEWIP  program  with  revising  their  approved  SEP  in  order  to  update  it  to  reflect 
programmatic  changes  that  occurred  during  the  SEP  approval  process. 

The  ALMDS  Program  Manager  has  concurred  and  agreed  to  develop  an  ALMDS 
Systems  Engineeering  Plan  and  forward  it  for  approval  by  March  2007.  This 
ALMDS  SEP.  will  be  an  annex  to  a  Global  OAMC’M  Systems  Engineering  Plan 
that  covers  the  MH-60S  helicopter,  the  “common  console,”  and  all  common 
interfaces.  This  approach  has  been  planned  with  and  approved  by  ASN  (RD&A) 
(CHENG).  ASN  (RD&A)  (CHENG)  will  work  with  the  AN/AES-1  Airborne 
Laser  Mine  Detection  System  program  to  obtain  approval  of  then  SEP. 

In  accordance  with  NAE  direction  of  27  April  2006.  ASN(RD&A)(C'HENG)  has 
been  delegated  the  responsibility  for  approval  of  all  AC  AT  IC.  LAC’  and  II  System 
Engineering  Plans.  Additionally,  SEP  Component  approval  authority  was  also 
delegated  for  ACAT  ID  and  LAM  programs  to  ASN(RD&A)(CHENG). 

Recommendation  A. 5: 

“We  recommend  that  the  Navy  Advanced  Extremely  High  Frequency  Multiband 
Terminal  program  manager  submit  an  updated  acquisition  program  baseline  to  the 
Navy  Acquisition  Executive  within  90  days  of  a  program  deviation.” 

DON  Comments: 

The  requirement  for  a  new  NMT  APB  was  due  to  the  increase  in  the  scope  and 
cost  of  the  program  to  a  new.  higher  ACAT  level  (from  an  ACAT  II  to  an  ACAT 
IC).  The  program  reported  an  APB  deviation  and  continued  to  resubmit  Program 
Deviation  Report’s  (PDR)  to  update  the  status.  Development  and  approval  time  of 
the  APB  was  unusually  extensive.  The  causative  factors  contributing  to  the 
lengthy  APB  update  were  as  follows:  (a)  waiting  for  USD(AT&L)  to  adjudicate 
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the  ACAT  designation  and  promulgate  the  ACAT  listing;  (b)  development  of  an 
Independent  Cost  Estimate  (ICE);  (c)  establishment  of  a  Consolidated  Acquisition 
Reporting  System  (CARS)  account  and  creation  of  NMT  CARS  file;  and  (d) 
reconciliation  of  original  APB  (not  in  CARS  format)  to  the  new  CARS  APB  and 
citing  derived  requirements.  (Note:  Any  one  of  the  aforementioned  events  could 
not  have  occurred  within  90  days  of  PDR  submission). 

The  draft  report  includes  an  incorrect  statement  (pg  15):  "However,  after  the 
program  was  designated  as  an  ACAT  IC.  the  approval  of  the  acquisition  program 
baseline  was  withdrawn.’’  The  Navy  proposes  this  comment  be  changed  to:  “After 
designation  of  the  program  as  an  ACAT  IC.  the  APB  revision  was  approved. 
However,  after  the  approval  it  was  noted  that  inadvertently  there  were  some 
changes  to  the  performance  section  in  error  that  required  the  APB  to  be  corrected 
and  resigned.  The  new  collected  revision  of  the  APB  has  been  forwarded  to 
A$N(RD&A)  for  approval.” 

While  the  APB  was  being  developed  the  program  office  continuously  provided 
updated  status  through  PDR's  to  ASN  (RD&A)  to  ensure  that  the  NAE  was  kept 
informed  on  the  status  of  the  revisions  needed.  These  PDRs  and  updates  were  in 
keeping  with  the  guidance  provided  in  the  DoN  Acquisition  and  Capabilities 
Guidebook  which  states:  "Within  90  days  of  the  program  deviation,  the  PM  is 

to . or  3)  provide  a  date  by  which  the  new  APB  will  be  submitted  or  by  which  the 

program  will  be  back  within  original  APB  thresholds.  4)  Keep  the  Chief  of  Naval 
Operations/Commandant  of  the  Marine  Corps  (DC.  P&R.  and  CD)  informed  with 
regard  to  program  deviations  and  baseline  recovery  actions." 

Recommendation  B.  1 : 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  require  program  offices  to  schedule  and  complete  an  operational 
assessment  prior  to  the  low-rate  initial  production  milestone  decision  in 
accordance  with  DoD  Instruction  5000.2.” 

DON  Comments: 

The  Navy  believes  its  policy  regarding  OA  activities  prior  to  LRIP  is  consistent 
with  the  DoDI  5000.2: 

Per  SECNAVINST  5000.2C: 

5.6.2  Navy  Procedures  for  Certification:  The  SYSCOM  Commander /Program 
Executive  Officer  (PEO)  Direct  Reporting  Program  Manager  (DRPM)  PM  shall 
convene  an  OTRR  prior  to  certifying  readiness  for  OT&E  (including  early 
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operational  assessment  (EOA),  OA.  IOT&E  OPEVAL.  SQT.  and  FOT&E).  The 
OTRR  shall  consist  of  all  members  of  the  testing  team  (DT&E  and  OT&E) 
includmg  representatives  from  C'NO  (N091),  the  program  sponsor.  Assistant 
Secretary  of  the  Navy  (Research.  Development  and  Acquisition)  (ASN(RD&A)) 
Chief  Engineer  (CHENG),  and  COMOPTEVFOR. 

5.6.4. 1  Waivers:  The  term  "Waivers"  applies  to  a  deviation  from  the  criteria 
identified  for  certification  in  paragraph  5.6.1  of  this  instruction.  Waivers  do  not 
change  or  delay  any  testing  or  evaluation  of  a  system. 

5. 6.4.2  Deferrals:  The  term  "Deferrals"  applies  to  a  delay  in  testing  requirements 
directed  by  the  TEMP.  A  deferral  moves  a  testing  requirement  from  one  test 
period  to  a  later  period.  Deferred  items  cannot  be  used  in  the  analysis  to  resolve 
COIs;  however,  the  OTA  may  comment  on  operational  considerations  in  the 
appropriate  sections  of  the  test  report.  A  deferral  does  not  change  the  requirement 
to  test  a  system  capability,  function,  or  mission,  only  the  tnneframe  in  which  it  is 
evaluated. 

5. 6.4. 2.1  When  Deferrals  are  Appropriate:  Deferrals  will  not  normally  be  granted 
for  EOAs.  OAs,  or  any  OT&E  prior  to  IOT&E  Performance  shortfalls  should  be 
identified  sufficiently  early  to  document  system  capability'  maturity  in  the 
appropriate  C'DD.  CPD.  and  TEMP.  When  unanticipated  problems  with  system 
maturity  or  test  resources  would  unduly  delay  an  OT  period,  deferrals  provide  for 
continued  testing  and  efficient  use  of  scheduled  resources  (e  g.,  ranges,  operational 
units,  and  assets). 

5.6.5  Navy  Waiver  and  Deferral  Requests:  Waivers  and  deferrals  shall  be 
requested  in  the  OT&E  certification  message.  If  a  waiver  or  deferral  request  is 
anticipated,  the  PM  shall  coordinate  with  the  program  sponsor.  CNO  (N912),  and 
COMOPTEVFOR  prior  to  the  OTRR  or  similar  review  forum.  Deferrals  shall  be 
identified  as  early  as  possible,  normally  no  later  than  30  days  prior  to  OTRR.  Use 
of  the  T&E  WIPT  or  similar  forum  is  also  recommended  to  ensure  frill 
understandmg  of  the  impact  on  operational  testing. 

When  requesting  a  waiver  or  deferral,  the  PM  shall  outline  the  limitations  that  the 
deferral  or  waiver  will  place  upon  the  system  under  test,  and  their  potential 
impacts  on  fleet  use.  Further,  a  statement  shall  be  made  in  the  OT&E  certification 
message  noting  when  approved  deferrals  will  be  available  for  subsequent  OT. 

The  Department  of  the  Navy  (DON)  Acquisition  and  Capabilities  Guidebook, 
states  that  the  MDA  should  determine  if  OT&E  is  required  prior  to  LRIP  for  non- 
OSD  T&E  oversight  programs.  The  SECNAV  guidance  therefore  (for  non-OSD 
T&E  oversight  programs)  gives  the  MDA  the  necessary  discretion  to  determine  if 
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Developmental  or  Integrated  Test  data  and  reports  are  adequate  to  make  an  LRIP 
decision  or  if  additional  OA  is  needed.  Forcing  such  programs  to  include  an  OA 
in  the  T&E  strategy,  when  program  risk  can  be  adequately  managed  without  the 
use  of  an  OA,  is  inefficient,  runs  counter  to  the  CNO’s  challenge  to  the  Navy  to 
streamline  test  and  evaluation.  Instead  this  guidance  allows  the  necessary 
flexibility  to  achieve  cost,  schedule  and  performance.  Current  guidance  is 
adequate  and  should  not  be  changed 

Recommendation  B.2: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  document  in  the  acquisition  decision  memorandum  justifications 
for  exceeding  the  10  percent  low-rate  initial  production  limitation.” 

DON  Comments: 

Concur,  with  comment.  ASN(RD&A)  will  document  in  the  acquisition  decision 
memorandum  the  justification  for  exceeding  the  10  percent  low-rate  initial 
production  guideline.  ASN(RD&A)  already  has  been  reflecting  this  justification 
in  recent  acquisition  decision  memorandums  and  the  statutorily  required  Selected 
Acquisition  Reports  (SARs)  and  will  continue  to  do  so. 

Comment:  The  Low-Rate  Initial  Production  (LRIP)  statute  does  not  set  10  percent 
as  “a  limitation”.  10  USC  §  2400  (b)  defines  LRIP  as: 

. . .  production  of  the  system  in  the  minimum  quantity  necessary— 

(1)  to  provide  production-configured  or  representative  articles  for 
operational  tests  pursuant  to  section  2399  of  this  title; 

(2)  to  establish  an  initial  production  base  for  the  system;  and 

(3)  to  permit  an  orderly  increase  m  the  production  rate  for  the  system 
sufficient  to  lead  to  full-rate  production  upon  the  successful 
completion  of  operational  testing. 

The  “10  percent  '  reference  comes  from  10  USC  §  2400  (a)(5)  which  establishes 
the  requirement  for  the  Secretary  of  Defense  to  include  a  statement  of  the  LRIP 
quantity  in  the  first  Selected  Acquisition  Report  (SAR).  10  USC  §  2400  (a)(5) 
states  that  “[i]f  the  quantity  exceeds  10  percent  of  the  total  number  of  articles  to  be 
produced,  as  determined  at  the  milestone  B  decision  with  respect  to  that  system, 
the  Secretary  shall  include  in  the  statement  [in  the  SAR]  the  reasons  for  such 
quantity.”  The  10  percent  is  not  a  limitation  but  is  instead  a  guideline  and  a 
threshold  for  which  further  explanation  and  documentation  in  the  SAR  is  required 
for  ACAT  I  programs.  In  addition,  it  should  be  noted  that  the  reasons  for 
exceeding  10  percent  are  also  typically  documented  and  thoroughly  staffed  in  the 
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Acquisition  Strategy  which  is  signed  by  the  Milestone  Decision  Authority  (MDA) 
prior  to  the  milestone  B  decision. 

Recommendation  B.3: 

"We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  approve  a  program  office's  request  to  exceed  the  10  percent  low- 
rate  initial  production  limitation  only  with  Commander.  Operational  test  and 
Evaluation  Force  recommendation  that  the  system  is  ready  to  proceed  to  low-rate 
initial  production.” 

DON  Comments: 

Do  not  concur.  As  described  under  B.2.  above,  10  USC  §  2400  does  not  establish 
10  percent  as  a  "limitation".  Additionally,  the  LRIP  statute  requires  both  the 
MDA  to  establish  the  LRIP  quantity  at  MS  B  and  the  Secretary  of  Defense  to 
report  on  the  LRIP  quantity  in  the  first  SA  (for  AC  AT  I  programs).  Both  of  these 
are  events  that  typically  occur  well  before  'the  system  is  ready  to  proceed  to  low- 
rate  initial  production”  as  suggested  in  the  recommendation. 

Additionally.  Commander,  Operational  test  and  Evaluation  Force's  (COTF  s) 
evaluation  of  a  system  is  restricted  to  whether  or  not  the  system  under  test  satisfies 
the  requirements  of  the  capabilities  documents.  Test  and  evaluation  is  only  one 
consideration  by  the  Milestone  Decision  Authority  (MDA)  in  support  of  a 
production  decision.  COTF  has  neither  connection  to.  nor  understanding  of.  all 
acquisition  considerations,  such  as  system  producibility,  funding  considerations, 
logistics,  contract  considerations,  etc..  Therefore.  COTF  is  unable  to  render  a 
complete  and  informed  recommendation  to  the  MDA  concerning  LRIP.  and 
should  limit  its  reporting  to  system  performance,  leaving  acquisition  officials  to 
resolve  production  decisions.  Production  decisions  should  remain  the 
responsibility  of  the  MDA. 

Recommendation  B.4: 

We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  instruct  the  program  manager  for  AV-SB  Harrier  II  Open 
Systems  Core  Avionics  Requirements  to  provide  documentation  that  the  suitability 
issues  m  the  Operational  Test  Report  for  the  AV-8B  Harrier  II  Open  Systems  Core 
Avionics  Requirements  were  collected  and  that  the  collections  were  verified  by 
the  Commander,  Operational  Test  and  Evaluation  Force.” 


82 


Final  Report 
Reference 


DON  Comments: 

Action  already  taken  and  clarification  is  provided.  There  were  four  major 
documentation  deficiencies  identified  in  the  OT  report  for  OSCAR.  By  agreement 
with  COMOPTEVFOR.  these  deficiencies  were  re-evaluated  during  subsequent 
operational  testing  of  the  H2.0  program.  The  COMOPTEVFOR  report  on  the 
H2.0  program  (COMOPTEVFOR  3980  (S0195-08-OT-IIIA-SCS-H2.0)  Ser 
573/S001  of  9  Jan  06)  stated  that  the  four  major  deficiencies  in  OSCAR  were 
corrected,  and  there  were  no  documentation  deficiencies  with  the  H2.0  program. 


Recommendation  B.5: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  include  in  acquisition  decision  memorandums  exit  criteria  that 
Commander.  Operational  Test  and  Evaluation  Force  testing  include  a 
recommendation  that  the  system  is  ready  to  proceed  to  low-rate  initial 
production.” 

DON  Comments: 

Do  not  concur.  Commander.  Operational  test  and  Evaluation  Force's  (COTF's) 
evaluation  of  a  system  is  restricted  to  whether  or  not  the  system  under  test  satisfies 
the  requirements  of  the  capabilities  documents.  Test  and  evaluation  is  only  one 
consideration  by  the  Milestone  Decision  Authority  (MDA)  in  support  of  a 
production  decision.  COTF  has  neither  connection  to.  nor  understanding  of.  all 
acquisition  considerations,  such  as  system  producibility,  funding  considerations, 
logistics,  contract  considerations,  etc..  Therefore.  COTF  is  unable  to  render  a 
complete  and  informed  recommendation  to  the  MDA  concerning  LRIP.  and 
should  limit  its  reporting  to  system  performance,  leaving  acquisition  officials  to 
resolve  production  decisions.  Production  decisions  should  remain  the 
responsibility  of  the  MDA.  No  change  should  be  made  to  the  current 
responsibilities  of  COTF  and  the  MDA.  Current  COTF  reporting  is  adequate  and 
should  not  be  changed. 


Recommendation  B.6: 

"We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research.  Development, 
and  Acquisition)  require  the  program  manager  for  the  AN  APG-79  Active 
Electronically  Scanned  Array  Radar  System  to  obtain  a  determination  from  the 
Commander,  Operational  Test  and  Evaluation  Force  that  the  AN  APG-79  Active 


Deleted 
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Electronically  Scanned  Array  Radar  System  is  potentially  operationally  effective 
and  suitable  before  approving  any  further  production  buys.” 

DON  Comments: 

Do  not  concur.  The  AESA  Operational  Evaluation  Test  completed  in  Nov  2006 
and  the  report  will  be  published  February  2007.  The  Beyond  LRIP  Report  will  be 
provided  in  March  2007  with  the  Full  Rate  Production  decision  planned  for  April 
2007.  The  proposed  terminology  "potentially  operationally  effective  and  suitable” 
would  be  applicable  to  an  Operational  Assessment  report  but  not  an  Operational 
Evaluation  Test  report.  Additionally,  in  the  LRIP  4  ADM  the  following  is  already 
an  exit  catena  -  “OPEVAL  Report  with  system  found  Operationally  Effective  and 
Suitable”. 


Recommendation  B.7: 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research.  Development, 
and  Acquisition)  instruct  the  program  manager  for  the  AN  SPY-ID(V)  Radar 
Upgrade  System  to  provide  documentation  that  the  suitability  issues  in  the 
Operational  Test  Report  for  the  AN.  SPY-1D(V)  Radar  Upgrade  System  were 
collected  and  that  the  corrections  were  verified  by  the  Commander,  Operational 
Test  and  Evaluation  Force.  In  addition,  we  recommend  that  closeout  of  the 
program  not  be  approved  until  the  requested  documentation  is  provided.” 

DON  Comments: 

Concur  with  comment.  The  ASN(RD&A)  letter  of  29  September  1993  fonnally 
approved  the  AN  SPY-ID(V)  Radar  Program  begmumg  at  Milestone  (M/S)  II 
under  the  prior  DOD  5000  process.  The  program  is  currently  under  LRIP  in  the 
old  acquisition  process  and  not  Full  Rate  Production. 

Prior  to  LRIP  approval,  an  independent  AN/SPY- 1D(V)  assessment  committee 
■was  tasked  by  ASN(RD&A)  to  ‘  assess  programmatic  and  technical  risk;  assess 
adequacy  of  land-based  testing;  assess  performance  of  SPY- ID  and  SPY-ID(V) 
against  current  and  near-tenn  threats;  and  compare  SPY-ID(V)  with  State-of-Art 
Technology”.  On  19  December  1994,  the  committee  briefed  the  final  report  to 
ASN(RD&A)  and  the  primary  recommendation  was  to  “complete  development 
and  deployment  of  the  AN/SPY-1D(V)  radar  without  delay”  Additionally,  a 
successful  land-based  operational  test  (OT-IIF1)  was  conducted  in  May  1996  at 
Combat  Systems  Engineering  Development  Site  in  Moorestown,  NJ.  The 
AN.  SPY-ID(V)  system  was  found  to  have  the  potential  to  be  operationally 
effective  and  suitable. 
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During  the  AN/SPY- 1D(V)  DoD  IG  audit  in  February  2006,  the  Program  Manager 
anticipated  a  successful  OPEVAL  to  permit  program  closeout  (i.e..  system  found 
to  be  “effective  and  suitable”)  The  Operational  Test  report  was  issued  on  5  April 
2006  and  the  program  didn't  meet  suitability  threshold  requirements.  The 
Program  Manager  is  implementing  a  plan  to  resolve  outstanding  suitability 
deficiencies  identified  in  the  OPEVAL  report  prior  to  Milestone  III  (FRP)  decision 
review. 


Recommendation  C.  1 : 

“We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  issue  official  guidance  to  Deputy  Assistant  Secretaries  of  the 
Natty,  program  executive  officers,  and  program  managers  mandating  the  use  of 
Dashboard  with  quarterly  updates  for  all  AC  AT  programs.” 

DON  Comments: 

Concur.  ASN(RD&A)  will  issue  official  guidance  to  the  Deputy  Assistant 
Secretaries  of  the  Navy.  Program  Executive  Officers,  and  the  Program  Managers 
mandating  the  use  of  Dashboard  requiting  quarterly  updates  for  all  AC  AT 
programs. 

Recommendation  C.2: 

“We  recommend  that  the  Assistant  Secretary  of  the  Natty  (Research.  Development, 
and  Acquisition)  modify  Dashboard  to  distinguish  different  increments  for 
evolutionary  acquisition  programs.” 

DON  Comments: 

Concur.  ASN(RD&A)  will  develop  the  needed  modifications  to  Dashboard  in 
order  to  allow  the  various,  different  increments  of  evolutionary  acquisition 
programs  to  be  distinguished  and  reported. 
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Revised 


DEPARTMENT  OF  THE  NAVY  RESPONSE  TO  VARIOUS  FINDINGS  AND 
COMMENTS  REGARDING  DODIG  DRAFT  REPORT  ON  NAVY 
ACQUISITION  EXECUTIVE’S  MANAGEMENT  OVERSIGHT  AND 
PROCUREMENT  AUTHORITY  FOR  ACQUISTIION  CATEGORY  I  AND  II 
PROGRAMS  (PROJECT  NO  D2005-D000AS-023.000) 

Finding: 

AN  APG-79  Active  Electronically  Scanned  Array  Radar  System,  Page  9.  “'The 
NAE  approved  the  AN  APG-79  Active  Electronically  Scanned  Array  Radar 
System’s  acquisition  strategy  22  days  after  approving  the  program’s  acquisition 
decision  memorandum  for  Milestone  C.  Program  officials  stated  that  the 
document  was  discussed  and  submitted  at  the  Milestone  C  review  and  the  approval 
was  delayed  because  of  an  oversight  in  the  NAE  approval  chain." 

DON  Comments: 

This  is  misleading  in  that  there  was  no  "oversight  in  the  NAE  approval 
chain."  The  "discussion"  at  the  decision  meeting  was  whether  to  change  the  Time 
Critical  Parts  milestone  into  a  LRIP  4  decision  in  FY06.  ASN(RD&A)  decided  to 
make  that  change  to  the  program  of  record.  Upon  his  decision  at  the  meeting  to 
make  the  change,  the  program  office  was  directed  to  make  the  appropriate  changes 
to  the  document  and  resubmit  it  to  ASN(RD&A)  for  signature.  These  required 
revisions  caused  the  delay  between  the  Milestone  C  decision  review  and  when 
ASN(RD&A)  signed  the  revised  document. 

Finding: 

MK  XII  Mode  5  Identification  Friend  or  Foe  Upgrade.  Page  9.  'The  Mode  5 
acquisition  strategy  was  approved  6  months  after  the  NAE  approved  the 
acquisition  decision  memorandum  for  Milestone  B.  The  Deputy  Assistant 
Secretary  of  the  Navy  action  officer  stated  that  during  the  milestone  decision 
briefing,  the  NAE  requested  several  changes  be  made  to  the  program's  acquisition 
strategy  before  he  would  approve  it.  The  NAE  approved  to  proceed  to  the  next 
acquisition  phase  because  he  did  not  want  to  delay  the  Milestone  V  decision.  It 
took  several  months  for  the  NAE-requested  changes  to  be  re-staffed  and  the 
acquisition  strategy  finally  approved.  However,  program  office  personnel  stated 
that  the  delay  in  approval  of  the  acquisition  strategy  was  due  to  staff  schedules  at 
the  NAE  level  and  that  the  acquisition  strategy  never  changed  after  it  was 
submitted  at  the  milestone  review.  After  examining  the  acquisition  strategy  we 
determined  that  the  NAE  signature  was  the  only  one  dated  after  the  acquisition 
decision  memorandum  was  approved. 
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DON  Comments: 

The  changes  made  to  the  Acquisition  Strategy  were  the  result  of  the  Milestone  B 
decision.  The  necessary,  required  changes  were  made  and  the  document  was 
restaffed,  however,  the  document  did  not  need  to  be  resigned  due  to  concurrence 
with  the  changes.  Upon  concurrence  by  the  other  signature  authorities,  the 
document  was  then  sent  to  ASN(RD&A)  for  his  final  signature. 

Finding: 

Common  Link  Integration  Processmg,  page  7-8.  “SECNAV  Instruction  5000. 2C 
requires  the  AoA  be  approved  by  the  NAE.  the  Chief  of  Navy  Operations,  or  the 
Commandant  of  the  Marine  Corps,  but  makes  no  mention  of  the  PEO  bemg 
approved  to  sign  the  AoA.  The  AoA  was  approved  by  the  PEO  instead  of  the 
NAE  as  required  by  the  SECNAV  Instruction  5000.2C.  According  to  the  program 
staff,  the  NAE  advised  them  that  the  AoA  could  be  approved  by  the  PEO.  but  they 
could  not  provide  supporting  documentation.'’ 

DON  Comments: 

The  findings  as  stated  in  the  report  are  correct.  The  AoA  cites  para  6.4.1  of  SNI 
5000. 2C  as  putting  the  overall  responsibility  of  the  AoA  in  the  hands  of  the 
Program  Executive  Officer  (PEO)/Systems  Command  (SYSCOM) 

Commander  Direct  Reporting  Program  Manager  (DRPM),  or  ASN(RD&A).  and 
Chief  of  Naval  Operations  (CNO)?Commandant  of  the  Marine  Coips  (CMC).  The 
Program  Office  interpreted  that  as  providing  adequate  authority  for  the  PEO  to 
approve. 

Finding: 

An  APY-ID(V)  Radar  Upgrade,  page  49.  '  The  AN  SPY-1D(V)  system  reports 
through  the  PEO  for  Integrated  Warfare  Systems.  The  NAE  approved  the  system 
for  low-rate  initial  production  on  January  24,  1997.  The  AN/SPY-1D(V)  system 
was  approved  for  a  second  low-rate  initial  production  on  December  21, 2001.  The 
system  is  an  upgrade  to  the  AN/SPY-1D(V)  to  improve  performance  in  adverse 
environments  with  natural  and  electronic  countermeasures.  The  AN  SPY-iD(V) 
system  is  a  component  of  the  Aegis  weapon  system.” 

DON  Comments: 

Incorrect  statement.  The  AN  SPY-1D(V)  is  an  upgrade  to  the  AN  SPY- ID  system. 
The  statement  should  read  as  follows:  "The  system  is  an  upgrade  to  the  AN/SPY- 
1D  to  improve  performance  in  adverse  natural  environments  and  in  electronic 
countermeasures.” 


Page  59 
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Pages  25  & 
26 


Finding: 

Surface  Electronic  Warfare  Improvement  Program,  pages  9-10.  “The  Surface 
Electronic  Warfare  Improvement  Program  (SEWIP)  had  an  approved  acquisition 
strategy  for  Block  1A:  however,  the  strategy  was  incomplete  because  it  did  not 
include  a  section  on  human  systems  integration  or  its  components  as  required  by 
DoD  Instruction  5000.2.  DoD  Instruction  5000.2  requires  the  program  manager  to 
have  a  comprehensive  plan  for  human  systems  integration  m  place  early  in  the 
acquisition  process  to  optimize  total  system  performance,  minimize  total 
ownership  costs,  and  ensure  that  the  system  is  built  to  accommodate  the 
characteristics  of  the  user  population  that  will  operate,  maintain,  and  support  the 
system.  A  SEWIP  program  official  provided  a  separate  human  systems 
integration  plan  for  the  program;  however  it  was  developed  after  both  spirals  of 
Block  1A  were  approved  for  Milestone  C.  The  Deputy  Assistant  Secretary  of  the 
Navy  action  officer  stated  that  only  spiral  2  of  Block  1A  required  human  systems 
integration  planning.  The  action  officer  stated  that  instead  of  including  the  plan  in 
the  acquisition  strategy  for  Block  1  A.  the  program  staff  created  a  separate  plan  as 
an  independent  document  for  the  program  and  that  the  plan  was  completed  or  near 
completion  when  the  NAE  approved  the  Milestone  C  acquisition  decision 
memorandum  for  spiral  2  of  Block  1A.  Because  human  systems  integration  was 
not  developed  early  in  the  acquisition  process,  the  program’s  system  performance 
and  total  ownership  costs  may  be  affected,  along  with  optimized  total  system 
performance.” 

DON  Comments: 

An  independent  HSI  plan,  which  encompasses  the  entire  SEWIP  Program,  was 
completed  in  accordance  with  the  DODI  5000.2  of  12  May  2003,  and  approved  31 
January  2006.  Acquisition  documentation  was  approved  by  the  Milestone 
Decision  Authority  (ASN  (RD&A))  per  Memorandum  of  August  13,  2002,  subject 
“Acquisition  Decision  Memorandum  for  the  Surface  Electronic  Warfare 
Improvement  Program. 

Finding: 

AN  AES-1  Airborne  Laser  Mine  Detection  System,  page  20.  “Program  officials 
for  the  AN  AES- 1  Airborne  Laser  Mine  Detection  System  did  not  conduct  an 
operational  assessment  prior  to  the  NAE  approval  of  the  LRIP  acquisition  decision 
memorandum  on  June  14,  2005.  Program  officials  eliminated  the  program's 
operational  assessment  because  the  acquisition  decision  memorandum  did  not 
require  it  and  program  officals  could  demonstrate  the  system’s  readiness  to  deploy 
during  developmental  testing.  A  developmental  test  report  was  issued  in  April 
2005.  It  stated  that  the  target  depth  and  false  contact  density  performance 
parameters  were  waived  for  Milestone  C.  hi  addition,  the  target  depth  key 
performance  parameter  threshold  was  reduced  to  60  percent  of  the  threshold  depth 
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for  the  LRIP  decision.  In  an  acquisition  decision  memorandum  dated  June  14, 
2005.  the  NAE  approved  Milestone  C'  LRIP  although  the  system  had  shortfalls  in 
three  performance  parameters,  based  on  acceptance  of  the  shortfalls  by  the  Fleet 
and  Resource  Sponsor.  Further,  the  NAE  approved  the  program  for  LRIP 
quantities  greater  than  the  10  percent  threshold  without  an  explanation  m  the 
acquisition  decision  memorandum  as  required  by  SECNAV  Instruction  5000. 2C.” 

DON  Comments: 

Tlie  Program  Office  agrees  with  the  finding  that  the  AN  AES- 1  program  did  not 
conduct  an  operational  assessment  prior  to  receiving  approval  for  LRIP  but  notes 
that  the  program  did  conduct  the  testing  that  was  required  by  its  Acquisition 
Decision  Memorandum  for  an  LRIP  decision.  Specifically,  the  program 
conducted  developmental  testmg  concurrently  with  COMOPTEVFOR. 
COMOPTEVFOR’ s  participation  was  in  the  capacity  of  a  DT-Assist  as 
documented  by  their  Letter  of  Observation.  COMOPTEVFOR  ltr  Ser  563/306 
dated  10  May  2005. 


Finding: 

AV-8B  Harrier  II  Open  Systems  Core  Avionics  Requirements,  page  10.  “Program 
Officials  for  AV-8B  Harrier  II  Open  Systems  Core  Avionics  Requirements 
(OSCAR)  obtamed  a  command,  control,  communications,  computers,  and 
mtelligence  support  plan  (C4ISP)  waiver  from  Deputy  Assistant  Secretary  of  the 
Navy  (DASN  Space  and  C4I):  however  the  waiver  was  not  properly  approved. 
DOD  Regulation  5000. 2R.  "Mandatory  Procedures  for  Major  Defense  Acquisition 
Programs  and  Major  Automated  Information  System  Acquisition  Programs,”  June 
10,  2001,  states  that  an  NAE.  with  the  advice  from  the  appropriate  Chief 
Information  Officer,  may  waive  C4ISP  preparation  if  the  Requirements  Authority 
has  previously  waived  the  requirement  for  an  interoperability  key  performance 
parameter  (KPP)  in  the  operational  requirements  document  (ORD).  The  Deputy 
Assistant  Secretary  of  the  Navy  Command,  Control,  Communications,  Computers, 
and  Intelligence  and  Space  approved  a  waiver  for  the  program's  command,  control, 
communications,  computers,  and  intelligence  support  plan.  The  waiver  was 
granted  because  the  program  did  not  have  mteroperability  KPP  in  the  AV-8B 
Radar  and  Night  Attack  ORDs.  OSCAR  program  officials  currently  consider 
these  two  ORD  s  as  their  operational  requirements  documents.  However,  as  noted 
in  the  ORD  section  of  this  findmg  we  have  determined  that  OSCAR  should  have 
developed  a  program-specific  ORD.  hi  addition,  the  waiver  was  not  approved  by 
the  NAE  as  required  by  DOD  Regulation  5000.2-R.  As  a  result,  the  reason  for 
approving  and  the  approval  authority  for  the  waiver  were  invalid;  the  program 
should  have  developed  a  command,  control,  communications,  computers,  and 
mtelligence  support  plan.” 


Pages  10  & 
11 
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DON  Comments: 

Tlie  ORD  Consolidation  Letter  for  OSCAR  Ser  582/C'OOl.  dated  29  Jan  02.  along 
with  the  series  of  documents  which  it  referenced  and  validated,  properly 
established  the  requirements  for  the  OSCAR  program.  Tliese  requirements  did  not 
include  an  interoperability  KPP.  Furthermore,  the  OSCAR  program  had  no 
Information  Exchange  Requirement  external  to  the  AV -8B.  Therefore,  the  basis 
of  the  original  waiver  was  valid.  The  fact  that  the  waiver  was  signed  by 
DASN(C4I  AND  SPACE)  mstead  of  by  NAE  does  not  alter  the  fundamental 
validity  of  the  waiver  request  or  imply  that  the  program  should  have  developed  a 
C4ISP. 


Fuidmg: 

Open  Systems  Core  Avionics  Requirements,  page  11.  "Hie  program  sponsor  did  not 
develop  an  ORD  for  the  OSCAR  program,  however,  the  NAE  approved  the 
program  to  proceed  through  all  milestone  decisions,  including  full-rate  production, 
without  requiring  the  program  manager  to  develop  an  ORD.  As  a  result,  we  were 
unable  to  determine  the  warfighter  requirement  for  the  OSCAR  acquisition 
program  or  identify  the  capability  gap  that  the  OSCAR  acquisition  program  was 
intended  to  fill.  In  addition,  without  an  approved  ORD.  the  NAE  was  unable  to 
determine  the  operational  performance  requirements  and  parameters  for  the 
proposed  system.  As  of  August  2004.  the  program  officials  did  not  intend  to  issue 
an  OSC'AR-specific  ORD."’ 

DON  Comments: 

As  the  OSCAR  system  replaced  the  existing  legacy  mission  and  weapons 
capabilities  in  the  Night  Attack  and  Radar  variants  of  the  aircraft,  the  warfighter 
requirement  is  defined  in  OR  025-05-84  AY-SB  Night  Attack,  dated  18  Sep  84 
and  OR  224-05-89  AV-8B  Radar,  dated  8  Aug  88.  The  JDAM  weapon  capability- 
introduced  by  the  OSC  AR  upgrade  is  governed  under  OR  CAF  401-91-2-A  Jomt 
Direct  Attack  Munition,  dated  23  Aug  95.  The  correction  to  the  aircraft  to  fully 
implement  Haveqiuck  SINCGARS  capability  within  OSCAR  is  governed  under 
OR  566-88-00  ARC-210  Radio  System,  dated  25  Sep  00.  The  incorporation  of  the 
Litening  Targetmg  Pod  capability  is  governed  under  the  Operational  Requirements 
Letter  for  the  AV-SB  Litening  Targeting  Pod.  dated  20  Apr  01.  The  mcorporation 
of  the  Advanced  Mission  C  omputer  is  governed  under  OR  549-88-00  Advanced 
Mission  C  omputer  and  Displays,  dated  21  Mar  00.  Taken  together,  these 
documents  fully  defined  the  operational  performance  requirements  for  the  OSCAR 
upgrade.  The  ORD  Consolidation  Letter  for  OSCAR  Ser  582  COOL  dated  29  Jan 
02  specifically  defined  the  performance  parameters  of  the  OSCAR  upgrade  based 
upon  the  performance  requirements  defined  in  the  above  mentioned  ORDs. 
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Finding: 

MH-60R  Multi-Mission  Helicopter,  page  13.  'Even  though  the  Director. 
Operational  Test  and  Evaluation  had  not  approved  the  MH-60R  test  and 
evaluation  mater  plan  Revision  C.  the  NAE  approved  full-rate  production  on 
March  31,  2006.  Revision  C  of  the  test  and  evaluation  master  plan  addresses 
follow-on  operational  test  and  evaluation  activities  for  the  MH60R  upgrades  and 
enhancements  m  the  form  of  planned  product  improvements.  The  program 
manager  must  subnut  the  approved  test  and  evaluation  master  plan  as  part  of  the 
full-rate  production  decision,  review.  Commander,  Operational  Test  and 
Evaluation  Force  conducts  follow-on  operational  test  and  evaluation  during  the 
operations  and  support  phase.  Therefore,  the  test  and  evaluation  master  plan 
Revision  C  should  have  been  approved  prior  to  full-rate  production  approval. 
Because  the  Director.  Operational  Test  and  Evaluation  had  not  approved  the  test 
and  evaluation  master  plan  Revision  C,  the  NAE  could  not  be  assured  that  planned 
follow-on  operational  test  and  evaluation  was  appropriate  or  adequate.” 

DON  Comments: 

The  program  had  an  approved  TEMP.  Rev  B.  which  called  out  the  Operational 
testing  required  for  the  milestone  and  COTF  provided  the  necessary  approval  to 
support  the  aircraft  for  Full  Rate  Production  as  tested.  Also,  the  TEMP  was 
updated  and  submitted  for  approval  prior  to  the  milestone.  Revision  C'  of  the 
TEMP  had  been  approved  through  N91  and  by  the  MDA  and  had  been  briefed  to 
DOT&E  staff  pnor  to  the  milestone.  DOT&E  staff  agreed  with  the  scope  of 
follow-on  testing  and  was  in  attendance  at  the  milestone  review.  Additionally, 
follow  on  operational  testing  was  briefed  at  the  milestone  review  with  the  MDA. 
Thus  when  the  MDA  made  the  decision  to  proceed  with  Milestone  C.  the 
program's  future  test  and  evaluation  plans  were  known  and  it  was  determined  that 
the  test  and  evaluation  plans  were  adequate  to  proceed  with  into  the  next  phase  of 
the  program. 

Findmg: 

MH-60R  Multi-Mission  Helicopter,  page  21-22.  "Program  officials  for  the  MH- 
60R  did  not  complete  an  operational  assessment  prior  to  LRIP II  approval.  The 
program  changed  its  acquisition  strategy'  from  a  remanufacture  approach  to  a  new- 
built  approach  startmg  with  LRIP  II.  in  an  acquisition  decision  memorandum 
dated  March  14,  2002.  In  addition,  the  acquisition  decision  memorandum  required 
completion  of  an  operational  assessment  with  a  conclusion  that  the  MH-60R  was 
potentially  operationally  effective  and  suitable  prior  to  the  approval  of  LRIP  II. 

The  Commander.  Operational  Test  and  Evaluation  Force  conducted  an  operational 
assessment  of  the  MH-60R  from  June  2,  2003.  through  September  3,  2003. 
However,  prior  to  completion  of  the  operational  assessment,  the  program  office 
terminated  the  testing  due  to  poor  performance  of  the  radar,  electronic  support 
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measures,  and  acoustic  subsystems.  The  NAE  approved  the  program  for  LRIP II 
on  December  15,  2003,  even  though  an  operational  assessment  was  not  completed, 
as  required  by  DoD  Instructions  5000.2  and  the  acquisition  decision  memorandum 
dated  March  14,  2002.  In  addition,  no  justification  was  included  in  the  acquisition 
decision  memorandum  for  proceedmg  without  the  required  operational  assessment. 

Further,  program  officials  conducted  a  second  operational  assessment  from 
October  through  December  2004.  The  operational  assessment  report,  dated  March 
18,  2005.  stated  that  the  system  did  not  meet  all  threshold  values  for  operation 
effectiveness  and  operational  suitability.  Although  the  report  included  a  listing  of 
recommendations  that,  if  not  addressed,  could  result  in  the  failure  of  operational 
evaluation  testing,  the  Commander,  Operational  Test  and  Evaluation  Force 
recommended  continued  program  development.  In  addition,  the  NAE  approved 
LRIP  III.  in  an  acquisition  decision  memorandum  dated  April  5,  2005.  ” 

DON  C  omments: 

Even  though  the  Operational  Assessment  was  stopped  by  the  Program  Office. 
C'OTF  did  m  fact  report  their  findings.  The  Program  Office  addressed  each  of 
those  findings  with  the  NAE  and  the  plans  to  correct  them.  Several  options  to 
proceed  with  the  program  were  briefed  to  the  NAE  with  corresponding  changes  to 
the  exit  cntena.  Accordingly,  the  NAE  approved  the  program  for  LRIP  II  and  the 
resultant  acquisition  decision  memorandum  references  the  brief,  which  describes 
in  detail  the  findings  from  the  operational  assessment,  and  specifically  identifies 
near  term  criteria  for  the  program  to  meet  in  order  to  proceed  with  Advanced 
Procurement  for  LRIP  III. 

In  June  2003,  COTF  changed  their  policy  for  reporting  out  of  Operational 
Assessments  to  specifically  identify  the  nsk  areas  of  the  program  based  on  their 
findings.  In  doing  so.  they  now  provide  the  MDA  with  detailed  information  so 
that  a  more  informed  decision  can  be  made  and  that  direction  can  be  given  as  to 
where  to  focus  the  program  efforts  for  the  next  milestone 

Finding: 

Preparing  and  Updating  Program  Documentation  Finding,  page  3.  “Since  FY 
2000.  the  NAE  approved  10  of  the  13  programs  we  renewed  for  entry  into  the 
next  phase  of  the  acquisition  process  before  obtaining  all  required  or  properly 
approved  acquisition  documentation.  The  NAE  did  not  require  3  of  the  13 
programs  to  have  approved  and  updated  acquisition  program  baselme 
documentation  between  milestone  decision  reviews  when  significant  changes 
affected  the  programs.  These  conditions  occurred  because  the  NAE  did  not  adhere 
to  DoD  and  Navy  acquisition  policy  when  reviewing  and  approving  programs  for 
a  milestone  decision  review  and  because  the  NAE  was  not  tracking  program 
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deviation  reports  in  the  Navy  Dashboard  database.  In  addition,  it  was  nuclear  why 
the  NAE  allowed  programs  to  proceed  through  milestones  without  required  or 
properly  approved  acquisition  documentation.  No  formal  tailored  documentation 
agreements  existed  and  documentation  decisions  associated  with  each  milestone 
review  were  not  being  formally  documented  in  document  waivers,  tailoring 
agreements,  or  acquisition  decision  memorandums.  As  a  result,  the  NAE  did  not 
have  all  the  necessary  information  required  to  make  fully  informed  milestone 
decisions  and  to  take  appropriate  management  actions  between  milestone  decision 
reviews.” 

DON  Comments: 

Specifically  regarding  the  table  on  page  6  titled,  “Key  Documentation  of  the  1 3 
Navy  Programs”  indicates  that  Tactical  Tomahawk  does  not  have  an  Acquisition 
Strategy.  This  is  incorrect.  The  Acquisition  Strategy  to  support  the  Tactical 
Tomahawk  MSIII  decision  was  signed  by  ASN(RD&A)  on  12  September  2003. 
The  Tactical  Tomahawk  paragraph  on  page  9  states,  “The  Tactical  Tomahawk 
program  officials  stated  that  the  NAE  had  approved  the  acquisition  strategy,  but 
they  were  unable  to  locate  the  signature  page  for  the  document.  Because  a 
signature  page  could  not  be  located,  we  could  not  verify  that  the  document  was 
approved  by  the  NAE.”  The  ASR  signed  12  Sept.  03  by  ASN(RD&A)  was  an 
ASR/AP  for  the  MSIII  FRP  decision  On  the  first  page  it  states,  "The  ASR  and 
AP  were  combined  for  streamlining  purposes  prior  to  the  milestone  decision. 

After  Milestone  III  approval.  AP  updates  will  be  approved  by  Program  Executive 
Officer.  Strike  Weapons  and  Unmanned  Aviation."  The  Acquisition  Plan 
approved  signed  by  ASN(RD&A)  on  April  26.  2004  was  an  AP  update  for  105 
missiles  to  the  FRP  contract. 

The  Analysis  of  Alternative  paragraph  on  page  7  states  the  following  with  respect 
to  Tactical  Tomahawk.  “Program  officials  for  the  following  four  programs  were 
unable  to  provide  an  AoA  or  cost  and  operational  effectiveness  analysis.”  Tactical 
Tomahawk  is  a  product  improvement  building  upon  the  Tomahawk  Block  III 
program.  Similarly,  the  Block  III  program  was  an  improvement  upon  the  Block  II 
program.  Program  documentation  indicated  that  prior  to  the  Tomahawk  Block  III 
milestone  III  decision,  the  MDA  determined  that  a  COEA  will  not  be  required  for 
Milestone  III  unless  conditions  have  changed  sufficiently  so  that  previous  cost- 
effectiveness  determinations  are  no  longer  valid.  Accordingly,  the  AoA  column 
for  Tactical  Tomahawk  in  the  Key  Documentation  table  on  page  6  should  indicate 
“N/A”.  Also,  the  Tactical  Tomahawk  program  should  be  removed  from  the  list  of 
four  programs  that  could  not  provide  a  required  AoA  /  COEA  on  page  7. 
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